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Background 

Title I of the Elementary and Secondary Education Act, which became 
law in 1965, authorizes financial assistance to Local Educatipn Agencies 
(vLEAs) that have concentrations of economically disadvantaged children. The 
authorized funds are used to provide programs specifically designed to meet 
the special needs of educationally disadvantaged children. 

Title I is a categorical grant program that appropriates funds to 
State Education Agencies (SEAs) for distribution to their eligible and par- 
ticipating LEAs which, in turn, bear primary responsibility for designing, 
implementing, and evaluating local projects. In their applications to 
State Education Agencies, LEAs must provide assurances that they will comply 
with Title I regulations, guidelines, and program criteria. States, in turn, 
are required to provide assurances to the U. S, Office of Education (USOE) 
that they will monitor LEA compliance with Title I directives, furnish 
technical assistance to LEAs, and submit annual statewide program evalua- 
tion reports to USOE, 

Title I legislation requires annual or periodic evaluations of the pro- 
gram at each of the various levels of its administration. LEAs are required 
to evaluate their local programs and submit evaluation reports to their SEAs 
on an annual basis. States are required to aggregate local data and^ submit 



a periodic State Title I Evaluation Report to USOE. On the basis of Sta 



at^^ 



Reports, surveys conducted by the National Center for Educational/Statistics, 
and special studies, USOE is required to provide an annual report to Congress 
on the national impact of the program. 

It has become increasingly clear in recent years that, despite con- 
siderable effort, an adequate assessment of Title I has yet to- be accom- 
plished. The State Reports have not been successful in fulfilling their 
potential as a source of information on program effectiveness. This failure 
is due in part to the quality of the inputs from the LEAs, problems in the 
methods used to aggregate data across school districts, and differences 
from state to state with respect to both the information reported and the 



formats adopted for its presentation. Guidel ines provided by USOE were 
inadequate to achieve the objectives of standardized content and formats 
and were abandoned in 1971. 

National surveys were equally unsuccessful in producing valid data 
regarding the nationwide impact of the Title I program. Two such surveys 
(USDHEW/OE, 1970; & Glass, 1970) were able to obtain meaningful cognitive 
impact data on only 9% and 7.5% of the population served, respectively. 
These samples, unfortunately, were not only small, they were non-repre- 
sentative as well, and thus provided no useful picture of overall program 
effectiveness- 
Special studies have also proved inconclusive, and one recent attempt 
to synthesize all existing evaluation data relevant to Title I (Wargo, 
Tallmadge, Michaels, Lipe, & Morris, 1972) concluded: 

Analysis of all possibly relevant data sources immediately 
indicated that nationally representative and valid impact 
data are simply not available and that some data relating 
to participation and expenditures also suffer from severe 
limitations (p. 32) . 

Obviously, the failure of past attempts to obtain adequate evaluation 
data has not diminished the need for such information. On the contrary, 
there appears to be an increasing insistence that the program be meaning- 
fully evaluated- The Education Amendments of 1974 (P-L. 93-380), for 
example, specify that: 

"Sec. 151- (a) The Commissioner shall provide for independent 
evaluations which describe and measure the impact of programs 
and projects assisted under this title. Such evaluations may be 
provided by contract or other arrangements, and all such evalua- 
tions shall be made by competent and independent persons, and 
shall include, whenever possible, opinions obtained from program 
or project participants about the strengths and weaknesses of 
such programs or projects- 




War^o, M- J-, Tallmadge, G- K- , Michaels, D. D- , Lipe, D- , & Morris, S. J. 
ESEA Titlfe i_: A reanalysis and synthesis of evaluation data from 
fiscal year 1965 through 1970 - Palo Alto, Calif- : American Institutes 
for Research, March 1972, (AIR-27400-3/72-FR) 
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" (b) The Commissioner shall develop and publish stan- 
dards for evaluation of program or project effectiveness ia 
achieving the objectives of this title. 

" (c) The Commissioner shall, where appropriate, consult 
with State agencies in order to provide for jointly sponsored 
objective evaluation studies of programs and projects assisted 
under this title within a State. 

" (d) The Commissioner shall provide to State educational 
agencies, models for evaluations of all programs conducted under 
this title, for their use in carrying out their functions under sec- 
tion 143(a), which shall include uniform procedures and criteria 
to be utilized by local educational agencies, as well as by the 
Stgite agency. in the evaluation of such programs. 

" (e) The Commissioner shall provide such technical and 
other assistance as may be necessary to State educational agencies 
to enable them to assist local educational agencies in the 4evelop- 
ment and application of a systematic evaluation of programs in 
accordance with the models developed by the Commissioner. 

(f) The models developed by the Commissioner shall 
specify objective criteria which shall be utilized in the evaluation 
of all programs and shall outline techniques (such as longitudinal 
studies of children involved in such programs) and methodology (such 
as the use of tests which yield comparable results) for producing 
data which are comparable on a statewide and nationwide basis. 

The intent of this legislation seems clear — not only with respect to the 
objective of obtaining good evaluation data, but regarding the manner in 
which the data should be compiled as well. 

While the present study was initiated prior to the existence of the 
legislative mandate cited above, it was motivated by the same felt need for 
better information concerning the largest federal program providing support 
for education. The study was also undertaken in the belief that the state 
reporting syste^n constituted the single most viable source for obtaining the 
desired information. 

Normally, almost all Title I activities are evaluated in one way or 
another at the local level, and some type of evaluation report is passed up 
from LEAs to SEAs, It has generally also been true that where Title I. 
activities are directed at the basic skill areas of reading and math, the 
evaluations have included some form of achievement testing. Finally, it has ^ 
been the assumption of at least some investigators working in the field that 
evaluators and project administrators alike are sincere in their efforts to 
assess the impact of their Title I projects on participating 'pupils. 



13 



3 



Given these conditions and assumptions, it seemed both reasonable 
and appropriate to attempt to design an evaluation and reporting system 
suitable for nationwide implementation at local, district, state, and 
federal levels. It also seemed reasonable to expect that such a system, 
if adequately constructed and "installed," could satisfy many, if not most 
of the evaluation needs of the Title I program at all levels. 

Objectives of the Study 

From what has been said above, it should be clear that the single, 
over-arching objective of the present study was to develop an evaluation 
and reporting system that would provide meaningful information to those 
involved at each level of the process. Implicit in this goal, although 
perhaps not immediately obvious, were the several sub-objectiVes of (a) de- 
termining the amount and type of evaluation information needed at each level 
of the program's administrative hierarchy, (b) assessing the ability and 
willingness of education agencies at each level to report the information 
desired by the higher levels, and (c) devising evaluation techniques and 
procedures which could be implemented in Title I settings and which would 
yield meaningful, valid, and useful data. Unless each of these sub-objec- 
tives could be achieved with a high degree of success, it was clear that 
there could be little hope for the overall success of the research and 
development effort. 

, In pursuing these goals, it was clearly appropriate to examine past 
State Title I Annual Evaluation Reports. As a logical^ outgrowth of this 
activity, the USOE-prepared Work Statement called for evaluating these 
reports along several relevant dimensions, aggregating whatever valid 
and comparable data could be extracted from them, and examining any trends 
which might be found with respect to reporting practices and the content 
of reports. The activities associated with this objective, while of only 
indirect relevance to the design of a model evaluation and reporting 
system, consumed a very large proportion of the total contract effort. 
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Methodology and Results 



The task of determining the amount and type of evaluation information 
needed at each level of the Title I administrative hierarchy was accomplished 
somewhat more rigorously at the national than at other levels. Interviews 
were held with educational policy makers in order to learn their information 
requirements regarding Title I operations and effects. 

Among those interviewed were a small number of federal legislators and 
senior professional staff members of all of the legislative committees and 
subcommittees involved wit'h the ESEA Title I program. Most of the senior 
Titie I program personnel in the U. S. Office of Education, and many others 
in DHEW and USOE who have responsibility for Title I legislation, regulations, 
guidelines, program criteria^ and/or evaluation were interviewed as well. 
While the list of interviewees was certainly not all-inclusive — particularly 
with respect to Senators and Congressmen involved in educational legislation — 
it was felt to be sufficiently large and representative to reflect current 
thinking at the federal level. This view was reinforced by the substantial 
consensus observed among the interviewed policy makers on nearly all of the 
issues discussed. 

The interviews were unstructured to give the interviewers freedom to 
discuss any topic of concern and to prevent possible preconceptions of the 
research staff from shaping or limiting interview coverage. While this 
approach unquestionably had merit, it unfortunately precluded a precise 
tallying of attitudes regarding specific issues. It was evident, nonetheless, 
that for nearly all of those interviewed the single most pressing concern 
was for some sort of measure of program impact on the children served. The 
comment was frequently encountered that after sothe ten years of program 
operation, it seemed only reasonable to have some objective and representa- 
tive data regarding student outcomes. 

There was also almost universal concern for information about the 
number of children served, the adequacy with which their needs had been 
assessed, the services offered to them, and the costs associated with Title I 
projects and programs. While these concerns were expressed by almost every- 
one, it was clear that the continued existence of at least marginally ade- 
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quate information in these areas over the years had not produced the same 
feelings of urgency and frustration that emerged in discussions about im- 
pact data. 

A number of other concerns were brought up by one or more of those 
interviewed. For' the most part these concerns, related to what might be termed 
the conditions of success. There was interest in such issues as whether 
it was better to serve children in early grades rather than later in their 
school careers, whether there was a clear-cut relationship between per- 
pupil expehditures and gains, and whether the intensity and/or duration of 
Title I treatments were crucial to their success. The inputs and effects 
of Parent Advisory Councils was another significant area of concern, but it 
was brought up by fewer than fifty percent of those interviewed. 

The conclusions to be drawn from the policy interviews were clear 
and unambiguous. The highest priority in developing a model evaluation and 
reporting system should be given to the assessment of Title I impact on 
the participating students. Considerable effort should also be given to 
methods for collecting and reporting information on Title I participation, 
resource allocation, and needs assessment. Then, to the extent possible, 
means should be developed for identifying possible relationships between 
program input variables and student outcomes. 

No first-hand data were collected regarding the information needs of 
state, district, or local school personnel involved in the administration 
of Title I projects or programs. These needs were inferred from the contents 
of the many local, district, and state evaluation reports which were read 
during the course of the study. Since most of these reports were prepared 
in the absence of guidelines or requirements imposed by higher levels, it 
seemed reasonable to assume that they contained information which their 
originators considered relevant and useful. ' Not surprisingly, it was found 
that these reports contained much the same kind of information which was 
wanted at the federal level, although there were gross variations in the 
data breakdowns and reporting formats used. 

The task of assessing the ability and willingness of schools, districts, 
and states to report the amount and kind of information desired by each 
higher administrative level was also accomplished largely through careful 
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reading and analysis of evaluation reports. Nearly 200 State Title I 
Annual EvaWation Reports for fiscal years 1971, 1972, 1973, and 1974 were 
examined as were many local-level evaluations- The variation in reporting 
practices was extreme, and report content ranged from near zero to the 
inclusion of more information than could conceivably be useful in any large- 
scale aggregation- Unfortunately, while it was clear that some agencies 
were able and willing to report large amounts of highly detailed information, 
it seemed equally likely that others could not or would not- From the docu- 
ments alone, it was not possible to determine the amount and type of infor- 
mation it would be reasonable to expect all states to report - 

A second issue which could not be adequately assessed through review 
of existing documentation was the extent of influence which SEAs exert over 
the LEAs within their jurisdiction- The two questions together comprised 
the basis for telephone inquiries directed to fifty-four State Title I 
Coordinators^- Using an open-ended, unstructured interview technique, 
information was sought regarding the amount and type of data which each 
state felt was reasonable to report- Relationships between each SEA and its 
LEAs were also explored - 

In gross and somewhat oversimplified terms, the conclusions reached 
from the telephone interviews were that (a) standardization of reporting 
content and format appeared feasible in all areas except impact ; (b) while 
some states perceived no great problem in adopting statewide testing and 
evaluation practices, others felt local autonomy was essential; (c) some 
states were moving strongly toward the use of criterion-referenced tests 
for Title I evaluations v/hile others were equally strong advocates of 
standardized tests; and (d) the meaningfulness , accuracy, and validity of 
the reported data would be inversely related to the amount requested- These 
conclusions, coupled with the information expressed by the educational 
policy makers interviewed in Washington provided important design objectives 
for the model evaluation and reporting system- 
Review of the State Title I Annual Evaluation Reports was an important 



1 Coordinators in American Samoa, Guam, and the Trust Territory were not 
contacted. 



step in establishing design obj ectives f or an improved reporting system. 
This task, however, was undertaken in substantially more depth than would 
have been necessary had that been the only objective to be achieved. The 
review also served the following additional purposes: 



a. It provided a systematic examination of all State 
Title I Annual Evaluation Reports submitted for 
fiscal years 1970 through 1974 in terms of their 
content, the adequacy (representativeness) of the 
data they contained, the extent to which they 
addressed relevant policy issues, and their 
general readability. 

b. It involved the preparation of critical sum- 
maries of each report for use in providing 
feedback to the author states. 

m 

c. It examined trends (or lack of them) in 
reporting practices. 

d. It provided aggregations of data on Title I 
participation, resource allocation, and impact 
to the extent that the quality of the reported 
data and the comparability of reporting formats 
permitted. 



Once it was determined that the most important information need of 
educational policy makers in Washington was for impact data a^d that it 
would be undesirable, if not impossible, to get all SEAs and LEAs to adopt 
a single evaluation procedure, work began on the design of an evaluation 
and reporting system which could accommodate these constraints. Most im- 
portant, of course, was the requirement that the data generated through 
implementation of the system be reliable and valid. Unless that objective 
could be achieved, all others would become meaningless. It was also impor- 
tant to develop a means for aggregating impact data across reporting units 
so that useful summaries could be prepared. 

Two RMC reports developed in conjunction with an earlier USOE/OPBE 
contract documented the hazards and constraints associated with carrying 
out valid impact assessments of educational projects (Tallmadge & Horst, 



ERIC 



8 

15 



1974; Horst, Tallmadge, & Wood, 1975) . The latter of these reports also 
outlined several evaluation designs which could feasibly be implemented 
in Title I contexts and which would yield valid and interpretable results. 
These designs formed the basis for the impact assessment portion of the 
prototype system. 

The system was designed so that any one of three evaluation models 
could be selected at the state, local, or even at the classroom level while 
other models could be chosen at other sites. Furthermore, two of the three 
models could be used with either standardized achievement tests or other 
typejs of tests for which no normative data were available, (e.g., criterion- 
or objectives-referenced tests). The models were all constructed to yield a 
measure of treatment effect defined as the difference between observed 
post-treatment performance (posttest scores) and an estimate of what per- 
formance would have been without the special instructional treatment. The 
models differ only with respect to the manner in which this no-treatment 
expectation is generated. 

Since the models all produce the same kind of impact measure, it would 
be possible to aggregate data across models using a simple additive process 
if all evaluations used the same test instrument. Since, however, one of 
the design objectives was to permit different sites to use different tests, 
a means had to be devised for expressing scores from different tests in 
terms of a common metric. The metric which was chosen for this purpose c 
was the national distribution of achievement levels, . 

The publishers of standardized achievement tests always provide tables 
whereby raw test scores can be converted into indices (usually percentiles) 
which define the status of individuals or groups with respect to a nationally 
representative sample of their age or grade-level peers. Since an individual . 
or a group is likely to achieve a score approximately the same relative dis- 
tance above or below the national norm regardless of which particular stan- 

^ Tallmadge, G, K, , & Horst, D, P, A procedural guide for validating 

achievement gains in educational projects (Revised). Los Altos, Calif,: 
KMC Research Corporation, December, 1974, (Technical Report No, UR-240), 

Horst, D, P,, Tallmadge, G, K, , &Wood, C, T, A practical guide to 
measuring proj ect impact on student achievement , Report 573-586, Wash- 
ington, D, C, : U. S. Government Printing Office, 1975. 



dardized test is used (assuming the tests cover comparable subject matter 
areas), these status indicators can be meaningfully aggregated using 
appropriate statistical procedures, whereas such aggregation would be 
meaningless using raw test scores. With standardized achievement tests, 
then, simply converting raw scores to scores reflecting status with respect 
to the national distribution solves the problem of the common metric and 
enables the aggregation of impact data across different test instruments. 

The aggregation problem is somewhat more complex where criterion- 
or objectives-referenced tests are used since, generally, no nationally 
representative normative data are available for these instruments. Two 
of the three evaluation designs included in the system were suitable for 
generating a no-treatment expectation and, thus, a measure of the treat- 
ment effect using unstandardized tests. The resulting measure, however, 
would be expressed in terms of raw scores rather than the desired common 
metric. To convert the raw score gain to the common metric, it would be 
necessary to know how a nationally representative sample of children would 
perform on the test. However, since' unstandardized tests have by defini- 
tion not been administered to nationally representative samples, it was 
necessary to devise a means of estimating the performance of such groups 
from other data sources. 

The estimation technique developed to serve this purpose required that 
scores be collected for treatment group children on a standardized achieve- 
ment test as well as on the unstandardized instrument used for pre- and 
posttesting. The second step of the estimation process entailed calcu- 
lating measures of the variability of scores (standard deviations) obtained 
by the treatment group on the standardized test as well as on the un- 
standardized post test. The standard deviation of the nationally repre- 
sentative sample on the standardized test was next obtained from the test 
publisher's manual. Under the assumption that that the ratio of the 
standard deviation of scores for the treatment group to the standard 
deviation of scores for the nationally representative sample would be the 
same for the unstandardized as for the standardized test, it was then a 
simple matter to compute the needed estimate of the performance (standard 
deviation) of a nationally representative sample on the unstandardized test. 
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Once this estimate was calculated, measures of treatment effects 
obtained from unstandardized tests could be expressed in terms of the same 
comraon metric as that devised for use with standardized achievement test 
scores. To do so required only that the raw score treatment-^ef f ect measure 
be divided by the estimated standard deviation for a nationally representa- 
tive sample. 

Once the common metric was developed, a few additional steps were 
taken to convert the scores into a form which was both easier to work with 
and easier to interpret than that which resulted directly from the com- 
putational procedures described above. The final common-metric scale has 
the same range (1 to 99) and midpoint (50) as the percentile scale and has 
the additional statistically desirable characteristic of possessing equal- 
intervdl score values (a characteristic not shared by the percentile scale) . 
Values on the final, transformed scale were given the name. Normal Curve 
Equivalents (of percentiles) or NCEs. They can be interpreted in much 
the same way as percentiles since, on the average, one NCE equals one 
percentile. 

After the system was developed conceptually and appeared to meet all 
of the design objectives set for it, work began on laying out procedures 
for its implementation. Preliminary forms were developed for recording, 
analyzing, and reporting data on participation, expenditures, personnel and 
training. Parent Advisory Councils, and impact. Detailed instructions 
were also prepared to facilitate filling out the forms correctly and 
consistently. 

Separate sets of forms and instructions were developed for use at the 
project level, the LEA level, and the SEA level. Separate forms and in- 
structions were also prepared for each of the three evaluation models and 
for the variations which enabled two of the models to be implemented with 
unstandardized as well as with standardized tests. 

The preliminary forms and instructions were reviewed by representa- 
tives of USOE and the project's Policy Advisory Panel, and revisions were 
made in accordance with their recommendations. The system was then taken 
to nine SEAs selected to represent different evaluation situations and 
approaches. The system was presented and explained in detail to appro- 
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priate SEA personnel, and their connnents and suggestions were solicited. 
, A second revision was made to the system after the comments and suggestions 

received from the nine SEAs were reviewed by USOE and Advisory Panel per- 
sonnel. 

Summary and Conclusions 

Analyses were conducted of (a) the information needs of educational 
policy makers regarding ESEA Title I operations and impact and (b) the 
willingness and ability of educational agencies at different levels to 
. provide reliable, valid, and consistently formatted data relevant to these 
needs. Based on these analyses, design characteristics were established 
for a new Title I evaluation and reporting system which would meet as many 
of the expressed information needs of the policy makers as possible within the 
constraints stated or implied by the Agencies who would provide the data. 
Additional system design requirements were derived from technical con- 
siderations relating to education measurement and evaluation design. 

An evaluation and reporting system was developed which appeared to 
meet all of the design objectives established for it. It includes stan- 
dardized forms and procedures for reporting information on participation, 
expenditures, personnel and training, and Parent Advisory Councils. It 
also provides three evaluation designs, two of which can be implemented 
with either standardized or unstandardized test instruments, which will 
yield valid and comparable impact data. Finally,, it incorporates a method 
for expressing all impact data in terms of a common metric thus enabling 
the aggregation of information across schools, districts, and states while 
allowing freedom of choice at all levels as to the evaluation design and 
test instrument to be used. 

The evaluation and reporting system was presented to appropriate 
personnel in nine State Departments of Education and to an Advisory Panel 
representing additional SEAs, LEAs, and other groups concerned wl,th Title I 
evaluation. While many comments and suggestions for revision were received, 
it appeared in general that attitudes toward the system were very favorable. 
Several SEAs expressed an eagerness to adopt the system immediately and 
nearly all wanted to participate in a field test of the system which may 
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be undertaken in the future. These reactions were substantially more 
positive than had been anticipated at the beginning of the contract effort 
and were interpreted as indicators of success in achieving the study's 
objectives. 

A follow-on project has been initiated which will involve, among 
other things, further refinement of the developed system. Ways of ex- 
tending its flexibility have been identified , and work is currently in 
progress to enhance both the utility and the acceptability of the system • 
At the present time, the prognosis appears excellent both that the system 
will be widely adopted in the next few years and that its adoption will 
result in vastly improved Title I evaluation and reporting practices. 
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I. INTRODUCTION 



Since passage of its enacting legislation in 1965, Title I of the 
Elementary and Secondary Education Act has been the largest federal pro- 
gram providing support for education. Designed to serve educationally 
disadvantaged children in schools serving economically disadvantaged pop- 
ulations, the program has long been the center of continuing controversy 
encompassing many technical, political, and even ethical issues. 

The often heated debates which have arisen over the Title I program 
have stimulated many attempts to assemble evaluative information about it. 
However, as pointed out by Wargo, Tallmadge, Michaels, Lipe, and Morris 
(1972), all evaluations have had serious methodological deficiencies. 
Often they have amounted to no more than collections of anecdotal infor- 
mation from which no meaningful inferences could be drawn. The cited re- 
port concluded: 



Analysis of all possibly relevant data sources immediately indi- 
cated that nationally representative and valid impact data are 
simply not available and that some data relating to participation 
and expenditures also suffer from severe limitations [p. 32 ]. 

Obviously, the failure of past attempts to obtain adequate evaluation 
data- has not diminished the need for such information. On the contrary, 
there appears to be an increasing insistence that the program be meaning- 
fully evaluated. The Education Amendments of 1974 (P.L. 93-380), for 
example, specify that: 

"Sec. 151. (a) The Commissioner shall provide for independent 
evaluations which describe and measure the/ impact of programs 
and projects assisted under this title, siich evaluations may be 
provided by contract or other arrangements I and all such evalua- 
tions shall be made by competent and independent persons, and 
shall include, whenever possible, opinions obtained from program 
or project participants about the strengths and weaknesses of 
such programs or proje.cts. 

" (b) The Commissioner shall develop and publish stan- 
dards for evaluation of program or project effectiveness in 
achieving the objectives of this title. 

" (c) The Cpmmissioner..<g^rall, where appropriate, consult 
with State agencies in order provide for jointly spoipored 



' 14 

ER?C 24 



objective evaluation studies of programs and projects assisted 
under this title within a State- 

" (d) The Commissioner shall provide to State educational 
agencies i models for evaluations of all programs conducted under 
this title, for their use in carrying out their functions under sec- 
tion 143(a), which shall include uniform procedures and criteria 
to be utilized by local educational agencies, as well as by the 
State agency in the evaluation of such programs - 

" (e) The Commissioner shall provide such technical and 
other assistance as may be necessary to State educational agencies 
to enable them to assist local educational agencies in the develop- 
ment and application of a systematic evaluation of programs in 
accordance with the models developed by the Commissioner. 

" (f) The models developedjiby the Commissioner shall 
specify objective criteria which shall be utilized in the evaluation 
of all programs and^ shall outline techniques (such as longitudinal 
studies of children involved. in such programs) and methodology (such 
as the use of tests which yield comparable results) for producing 
data which are comparable on a statewide and nationwide basis - 

The intent of this legislation seems clear — ^not only with respect to the 
objective of obtaining good evaluation data, but regarding the manner in 
which the data should be compiled as well. 

While nationwide surveys have been attempted in the past (e.g., 
USDHEW/OE, 1970; Glass, 1970) they have been particularly unsatisfactory. 
The two national surveys, discussed by Wargo et al. (1972) covered fiscal 
years 1968 and 1969 and provided meaningful cognitive impact data on only 
9% and 7.5% of the population served respectively. These samples, of 
course, were nonrepresentative of the population as a whole and thus pro- 
vided no useful picture of overall prograd effectiveness. 

The usefulness of other data source^s was also assessed by. the Wargo 
et al. (1972) report which concluded thaty "State Title I Annual Evaluation 
Reports. . .represented the largest source of potentially useful data avail- 
able in a set of apparently homogeneous repotts [p. 34]." While it was 
clear from the analyses included in the study that the reports in no way 
lived lip to their potential, there is agreement in many circles that the 
potential does exist and substantially exceeds that of survey approaches. 
It was, in fact, the belief that much of this potential could be realized 
through improvement and standardization of state evaluations and reporting 
practices which led to initiation of the present study. It was the same 
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belief, presumably, whic-h led to the specific legislative foundations 
quoted above- Thus, while the research was initiated prior to the 
legislative mandate, agreement was high with respect to both objectives 
and approach. 

The objectives of this study were to summarize the overall impact 
of Title I programs to the extent possible from FY 69-74 State Title I 
Annual Evaluation Reports, atid to design an improved reporting system 
which would increase the amount and usefulness of data contained in 
future State Reports, Phases I and II of the research were primarily 
addressed to the first of these objectives while the third and fourth 
phases were directed toward the more important latter objective. 

The research and development activities of the study were all 
oriented toward the ultimate objective of achieving meaningful and valid 
nationwide assessments of Title I impact through optimization of state 
and local-level reporting systems. In working toward this objective, 
tlxe reports for 1969-74 were examined and analyzed for trends, and the feasi 
bility of alternative procedures and practices were investigated through 
direct contact with cognizant SEA personnel. The recommendations and 
model reporting jsystem resulting from these activities represent a con- 
certed attempt tp reflect system output priorities, the practical time 
and monetary constraints existing at each level of the reporting system, 
the technical measurement and sampling considerations required for drawing 
valid inferences, and even the personal preferences of evaluators at 
all levels to the extent that they can be accommodated without com- 
promising other fd^atures of the system. 

In the following chapters, the activities of each phase of the study 
are described in detail. At the end of each section, conclus,;Lons and 
recommendations resulting from that phase of the study are presented, 
although in many cases these recommendations were revised or dropped 
at a later poiiit in the study. This report, then, provides a record 
of all the processes which contributed to the model reporting system, in- 
cluding the particular activities or inputs which resulted in decisions 
about what types of information should be included in or excluded from 
the reporting models. 
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II. PHASE I 



Summary of Phase I Tasks 

The first phase of- this study encompassed six tasks. Most of them 
closely paralleled analyses undertaken by Wargo and his colleagues (1972) 
and were, in. fact, designed to permit comparison of findings across time 
periods covered by the two studies. The first task was to analyze the 
content of the State Title I Reports and entailed tabulating information 
on program participation, resource allocation, and impact. The goal of 
this task and the corresponding analyses conducted by Wargo et al. (1972) 
was to. compile whatever useful information the State Reports contained 
so as to synthesize a nationwide picture of Title I operation and impact. 

The first part of this study involved ma|cing a content analysis of 
the State Reports to identify the types of population characteristics 
which were, reported and to determine whether these characlzeristics could 
be aggregated across most of the states. The results of the study, how- 
ever, showed that there were serious discrepancies between the State Re- 
ports which precluded the aggregation of information across more than a 
few states at a time. A similar finding in the earlier study by the Ameri- 
can Institutes for Research' (AIR) led the authors to conclude; 

In summary, state reports provide only a minimum amount, of pop- 
ulation characteristic information that is coiaparable' across 
states and fiscal years., Also, those data that are reported 
are greatly reduced in value by the confusion as to whether their 
population counts are duplicated or linduplicated. Finally, as 
a result of the duplicated/unduplicated count problem, determina- 
tion of interrelationships between population characteristics 
and other variables across states are subject to inaccuracies that 
are indeterminable (Wargo, et al., 1972, p. 35.) 

The AIR investigators also examined the breakdowns for expenditures 
and for the typegf of evaluation designs and tests used by the states. Their 
results again showed that the states had few consistent procedures for 
reporting expenditures and evaluation programs. Hence, the content analysis 
conducted by AIR revealed that it was not possible to fulfill the national 
evaluatj.bn requirements of Title I because the information contained in 
most reports could not be aggregated across the states. The content 



17 



27 



analysis conducted in conjunction with the present study indicates that 
the states have not become significantly more consistent in terms of 
selecting similar population characteristics and evaluation techniques. 
This conclusion was reached by comparing the data from the last four 
years with the AIR data. 

The second task, evaluating^ the adequacy and validity of the reported 
data , also followed the earlier AIR work and made use of the ^Data Adequacy 
and Validity Scale employed in that s.tudy. The scale was used to assess 
the representativeness of ' the evaluation samples described in the reports 
and to determine the extent to which cognitive benefit information was 
related to target group characteristics, types of projects, and total- or 
per-pupil costs. The results of the AIR evaluation again showed that 
there were serious problems concerning the validity of the data reported 
by most states, and the authors concluded: 

Sampling considerations, unfortunately, were not the only factors 
limiting the usefulness of reported cognitive data. Variations 
in experimental design, types of scores reported, and analytical 
unit (e.g.,. grade level, project, regular/ summer) not only'made 
the pooling of information across states difficult or impossible 
but, in some cases, actually precluded any meaningful interpre- 
tation of the data presented (Wargo, et al., 1972, p. 174.) . 

The present study involved rating the states' FY 71, 72, 73 and 74 

reports and included a trend analysis across the six years' covered by the 

two studies. Again, the more recent reports reflected essentially the 

same problems as were described. by AIR, and there were no. identifiable 

trends toward more valid or consistent evaluation or reporting pr.actices 

across the years. 

The third task was determining whether important policy issues were 
discussed in the reports . Information about which issues were ''important" 
came from interviews conducted by RMC staf f ^ members with key officials 
in H.E.W. and U.S.O.E. as well as with federal-level legislators and their 
professional staffs'. • . . 

There was substantial agreement among the individuals interviewed as 
to which policy issues were the most relevant. In essence, the major 
concerns expressed were centered on the necessity of knowing how Title I 
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funds are being spent, how target groups have been defined, how needs of 
the groups have been assessed, and whether the treatments have resulted 
in significant educational gains. The greatest emphasis was placed on the 
last of these areas and most of the interviewees expressed a felt need 
for impact, data to be presented at the project level. 

The Policy Relevance Scale was constructed by listing the important 
issues and weighting them to reflect their relative importance to. the 
policy makers interviewed. The ratings associated with each listed issue 
were analyzed in terms of frequency counts and percentages of states which 
obtained scores at different points on the scale. Comparisons across 
years were also made. The majority of the reports examined presented 
little policy-relevant information. The evaluation criteria, however > 
were developed after the fact and no guidance had been provided to state s 
as to what issues they should address' in their reports. Low ratings on 
the Policy Relevance Scale, therefore, cannot be interpreted as either a 
disinclination or inability on the part of the states to provide data 
desired at the federal level . 

The final Phase I task was concerned with evaluating the readability 
of the State Reports ,^ A rating scale was developed which reflected 
stylistic guidelines from handbooks such as the APA Publication Manual (1974) 
and Strunk and White's Elements of Style (1959). The eight dimensions 
included in this scale were designed to assess both the literary style and 
the thoroughness of the technical descriptions. All available State Reports 
for FYV71, 72, 73, and 74 were rated on the scale, and reports generally 
received high ratings. 

The content analysis and rating data which are presented in the 
following chapter were used to identify the major reporting problems which 
currently exist and which would have to be eliminated in order to pro- 
duce more meaningful assessments of Title I projects and programs. The 
analyses led to recommended changes in current evaluation and reporting . 
procedures which were then assessed in terms of acceptability, feasibility, 
and cost in later phases of ^the^t^idyC' It must be pointed out that the 
analyses des^rl^d^dLn-'tlris' following chapter were not used to criticize 
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previous reporting methods., but rather td improve the quality of future 
reports. Not only would it have been inappropriate to. assess reports 
in terms of criteria developed after they were written, but also no use- 
ful purpose could have been served by any kind of evaluative comparison 
among reports or states. 

Content Analysis j 
Methods 

When the FY 71, 72, 73, and 74 reports were reviewed, operational 
definitions were made for checklist categories in order to make the con- 
tent analysis consistent across reviewers. At the time the content of 
these reports was analyzed, definitions used in the review of FY 69 and 70 
reports in the AIR study (1972) were not available. After the analysis 
of the later reports was complete, a comparison of the data across all 
five fiscal years indicated that the later review had been done some- 
what differently from the review of the FY 69 and 70 reports. A search 
through the files at AIR produced documents which revealed several dif- 
ferences in analysis techniques and definitions between the AIR. review 
and the RMC review. The most important difference between the reviews 
was a difference in method. In the present study, credit was given for 
any data element which could be derived from others through summation or 
subdivision processes even if the element itself was not presented directly. 
For example, if a state only reported the number of Title I participants 
by grade levels but the total number could be calculated by suiraning across 
grades, the report was given credit for reporting the total number of par- 
ticipants. This was not done by the AIR reviewers, who only tallied data 
that were specifically labeled in the reports. This difference in method 
contributed to an unknown degree to the apparent increase in the reporting 
of unidimensional breakdowns from FY 69 and 70 to FY 71, 72, 73, and 74 
(see .Tables 1 through 3.) 

In the RMC review, a "program" was defined as a broad area of in- 
struction or service, such as readtog-j-^mathematics, health services, or i 
food. A "project" was defined as a unit of one or more such services under 
a single administration. In the AIR analysis, no attempt was made to adopt 
uniform definitions, and tallies were made in accordance with the words 
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used in the reports. .Since states used the terms interchangeably, the 
AIR tallies by program and project are virtually meaningless, and are not 
comparable to the RMC tallies. 

In the current review, the target population and expenditure data 
were considered as being reported by LEA when the reported breakdown 
appeared to be by LEA but wds not specifically defined as such. In the 
AIR review, LEA was marked only when the report stated that data were 
broken down by LEA. This difference prob^ably accounts for the apparent 
increase in breakdowns by LEA from FY 69 and 70 to FY 71, 72, 73, and 74. 

Despite these known differences in methods and definitions between 
the two analyses, and despite the possibility that unknown differences 
may also exist, it is possible to draw some conclusions about the content 
of the reports across the five fiscal years. Overall, the reports prob- 
ably have changed very little in content n 

Results 

•As can be seen from Table 1, almost every state reported the total 
population served for each of the six fiscal years. Figures for FY 74 
are somewhat lower than those for previous years because several reports 
from that fiscal year were not available for analysis. (See Appendix A 
for a list of reports which were analyzed for each fiscal year.) The most 
common unidimensional breakdowns in FY 71, 72, 73, and 74, as in FY 69 and 
70, were public/nonpublic participation, summer /regular year participation, 
participation by program, and participation by grade level. Although 
multidimensional breakdoms were used somewhat more frequently in the 
later reports, no such breakdown was used by more than nine states across 
all four years. 

As with the FY 69 and 70 reports, it was not always possible to 
determine for FY 71, 72, 73, and 74 whether participating children were 
counted more than once when they fell within more than one population break- 
down. ^ Figures in the reports were sometimes identified as duplicated 
counts and sometimes as unduplicated counts, but many times the methods used 
in obtaining the figures were not reported. Thus, for all six years, any 
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TABLE 1 

I^umber of State Reports Giving Breakdovffi o£ Target 
Population Served by Various Categories for Ff 69CW=46), 
70(N=45), 71CN=53), 72(N=52), 73(N=52), 74CN=30) 



Target Population 



Total 
Population 



Evaluation 
Sample 



Other 
Sample 



reported by: 


69 


70 


71 


72 


73 


74 


69 


70 71 


72 


73 


74 


69 


70 


71 


72 


73 74 


Geographic Units 


































State 


46 


43 


49 


46 


47 


25 








2 














County 


/ 

H 


1 




1 




1 




1 


















O^ilWJWJi^ J. O L> 1. J. ^ L> 


4 


3 


2 


2 


2 


2 


1 


1 


1 


1 






1 




1 


\ 


LEA 




3 


5 


4 


5 
























Other 


2 






1 


2 


3 






















Public-Nonpublic 


44 


42 


39 


33 


35 


16 






1 


1 


4 












Summer-Regular Year 


6 


4 


21 


17 


21 


12 








1 


2 












Ethnic Group 


5 


4 


7 


10 


12 


8 




1 1 


1 


2 










1 




Urban-Rural 


0 


1 


1 


1 


1 




4 


3 




1 














Fatnily Income 






1 










1 


















Program 


15 


14 


23 


28' 


28 


20 


1 


2 


2 


3 


1 


1 


6 








Project 


1 


3 


4 


2 




1 




1 


1 


1 








2 


1 


2 


Grade Level 


16 


14 


24 


23 


26 


21 


4 


6 2 


1 


5 


1 


2 


2 








firadp RanrI 




1 


6 


7 


8 


4 








1 














Interactions 


































Grade x county 


1 




1 








V 




















Grade x program 


o 
J 


/, 


o 
Z 


J 


z 


J 




9 


J. 


J. 


X 












Grade x project 




1 












1 


1 


1 














Grade x public-nonpublic 


8 


5 


9 


14 


16 


10 








1 














Grade x puo/non x sum/reg x prog 


1 


1 


3 


3 


3 


2 






















Grade x pub/non x sum/ regular 


1 


2 


8 


4 


4 


4 






















Grade x pub/non x school district 


1 
































Grade x summer/regular 


1 


2 


10 


7 


6 


6 






















Grade x urban-rural 














1 


2 


















Program x pub/non x summer /reg 


2 






1 


3 
























Program x pub/non x county 


1 










1 






















Program x summer/ regular 


1 




7 


10 


7 


6 








1 








1 






Public/non X Li£A 




1 


2 


2 


2 
























t'ublic/non x district 




1 








1 






















Public/non x summer/regular 




1 


5 


5 


7 


3 






















Summer/regular x race x county 


1 
































Other 






18 


10 


13 


5 




2 


3 


4 














No Data 




2 


1 


1 




1 























Note . Total population was defined as the total number of children served. The evaluation 

sample comprised the total number of children for whom impact data were reported. Other 
samples were nonrepresentative groups such as children in rural schools, those in the 
northeastern part of the state, etc. 
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aggregation of population data across states and across fiscal years would 
be influenced to an undeterminable degree by the mixing of duplicated 
and unduplicated counts. 

As in FY 69 and 70, data on expenditures in the Title I evaluation 
reports for the four later fiscal years were less complete than those 
dealing with the population served. A total of 42 reports over the four 
fiscal years presented no data at all on expenditures (See Table 2). In 
every case, per-pupil expenditures were reported less frequently than were 
total expenditures. For total expenditures, breakdowns of expenditures 
by program arid by specific instructional/specific supportive services could 
be aggregated across 13 and 4 states respectively. Presumably, aggregations 
could be made across a larger number of states if more 74 reports were 
available- Breakdowns by regular year/sximmer expenditures could be aggre- 
gated across five states. 

Table 3 shows the formats used by states in reporting cognitive bene- 
fits resulting from Title I projects. The category "Criterion-referenced 
Tests" was added for the current analysis. States were tallied in the "All 
Samples" portion of the table when there was no variation in evaluation de- 
signs within the state report. Thus, reports which presented only one sample, 
even if nbnrepresentative, were tallied in this column as well as reports 
which presented data on a number of samples using the same evaluation design. 
The "Some Samples" column was used to indicate those reports that presented 
data based on a variety of evaluation designs. 

In FY 70, 71. 72, 73, and 74, a few more states reported cognitive benefit 
data than did states in 19.69. As can be seen, sample characteristics varied 
greatly among states for all six fiscal years. In a total of 54 reports over 
the last four fiscal years, the method used in selecting the sample was not 
reported. It should be noted that in the RMC analysis, one sample often 
was marked in several categories in an effort to describe the selection 
process fully. If a state randomly selected certain projects and presented 
evaluation data for those projects, both "Random Selection" and "Selection 
by District/Project" would be marked. States frequently indicated that 
they included data only for projects or districts which had submitted their 
data before a certain deadline. In this case, "Other Selection Process" 
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TABLE 2 

Number of State Reports Providing Expenditures by Various 
Categories for FY 69(N=46), 70(N=45), 71(N=50), 72(N=52), 
73(N=52), 74(N=30) 



ESEA Title I expenditures 
reported by: 






Total 










PervPupil 






69 


70 


71 


72 


73 


74 


69 


70 


71 


72 


73 


74 


Geographic Units 


























State 


25 


22 


35 


38 


36 


18 


15 


9 


17 


16 


19 


9 


County 


4 


2 


3 


2 


4 


1 


2 










1 


School District 


6 


3 


3 


3 


3 


1 


2 


1 




1 


1 


2 


LEA 




2 


2 


3 


4 


1 








2 


2 




Other 






1 




1 


2 










1 




Program 


11 


14 


20 


25 


25 


13 


6 


9 


7 


8 




10 


Project 


2 


3 


8 


4 


2 


2 


1 


1 


3 




1 


1 


Instructional Services/Supportive Services 


15 


12 


17 


18 


19 


5 


1 


1 


1 


2 




3 


Specific Instructional and Specific 


























Supportive Services 


13 


12 


15 


17 


20 


4 


3 


3 










Regular Year/Summer 


5 


7 


7 


13 


11 


5 


1 


4 


6 


7 


6 


1 



Interactions 

Program x regular year/ summer 13 4 5 5 6 1 

Program x regular /summer x county 1 1 

Program x instruc/support x reg/summer 1 2 3 1 1 

Program x school district 1 

Specific instruc/support x county 1 

Specific support services x reg/sum 1 13 2 1 

Instructional/support services x reg/sum 15 5 4 1 

LEA X regular year/summer 111 
Elementary/secondary x county 1 

No Data . 17 15 13 11 11 7 

Other L 
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TABLE 3 



Number of States Providing Evaluation Data of Various Types 
for FY 69, 71, 71, 72, 73, and 74 



Evaluation Design 






All 


Samples 








^ome 


Samples 




69 


70 


71 


1 L 


















73 


74 


69 


70 


71 


72 


73 


74 


i%e- & posttest 


34 


35 


40 


39 


35 


19 


3 


3 


Q 


9 


7 


c 
3 


Posttest with norms 


2 


1 










2 


4 


2 


2 




1 
X 


Experimental/comparison 


2 


3 


2 


1 


1 




4 


9 




3 


3 




Criterion Referenced Tests 










1 


2 










3 


9 


Data Reported, design not specified 


2 


1 






2 


1 










4 


1 


Sample Characteristics 


























N given 


39 


37 


42 


35 


33 










6 


8 




Random Selection 


4 


2 




5 


4 


J 






1 




3 


2 


Selection by use of same tests 


15 


13 


10 


9 


6 


i; 


J. 


J. 




2 


1 


o 
Z 


Selection by district /project 


6 


9 


7 


5 


8 


1 n 

J- VJ 


0 
£. 




1 
J. 


3 


1 


1 


Sample of grade levels 


6 


6 




2 


2 


1 


Q 


1 
J. 


1 
J. 


1 


1 


1 


Selection by Program 


3 


2 


5 


3 


4 






1 
J. 






1 




Other selection process 


3 


3 


13 


14 


13 


o 






9 


3 


1 




Data reported, method not given 


12 


7 


17 


12 


18 


7 




J. 


9 


2 


2 


Z 


Score Breakdowns 


























State (no breakdowns) 


9 


9 




1 






2 


4 


1 




1 




County 






1 


1 


1 


















1 


1 


5 


3 


5 






1 


1 


2 




1 


LEA 


1 


1 


1 










2 










Urban-rural 


1 


1 


1 




1 










1 






Instructional program 


5 


4 


15 


16 


17 


18 


2 


1 


3 


5 


3 


1 


Project 


5 


1 


3 


2 


1 


2 




1 


6 


3 


3 


3 


Grade level 


24 


24 


20 


20 


22 


19 


6 


5 


11 


11 


10 


2 


Grade band 


3 


2 


1 


1 


1 


4 




1 


3 




2 




Public-nonpublic 


1 


2 


2 


3 


1 


2 


1 




2 


3 


2 




City size 


1 














1 










Regular year/summer 


2 


4 


4 


6 


6 


3 


1 


1 


3 


1 






Interactions 


3 


1 


12 


11 


10 








1 


5 


5 




Other 




3 


8 


7 


11 


6 


1 


3 


9 


10 


8 


3 
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TABLE 3 (cont.) 



Evaluation Design 



All Samples 



Sotee Samples 







/ u 


71 
/ J. 


7? 


7*^ 






70 


71 


79 


7*^ 




Type of Score Reported 


























Raw Scores 


4 


2 


2 


2 


5 


1 


5 


3 


4 


9 


5 


2 


Standard scores (K, T, Z) 


2 


1 


1 




1 




2 




1 


2 


4 


3 


Stanine 




1 














1 


2 


6 


1 


Percentiles 


2 


3 


6 


3 


1 




3 


4 


4 


5 


3 


2 


Deciles 


1 


1 








1 


1 






2 


1 




Quart iles 


11 


8 


3 


2 


3 


1 


2 


2 


5 


2 


1 




Grade equivalent 


12 


9 


4 


4 


2 


4 


5 


6 


6 


4 


5 


3 


Grade-equivalent gains 


14 


18 


29 


29 


31 


18 


6 


6 


14 


13 


12 


7 


Other 




6 






1 


2 


3 


7 


7 


3 


3 


4 


Summary Statistics 


























Mean 


22 


25 


21 


17 


24 


20 


9 


3 


15 


17 


14 


3 


Median 


4 


2 


3 


1 






2 




2 


2 






Mode 


1 


1 
















1 






Standard deviation 


2 


5 


2 




4 


2 


4 


3 


4 


4 


3 


1 


Percent upward shift 


3 




15 


15 


7 


3 


2 




13 


14 


15 


2 


Other 








1 










1 


1 






No Data 


5 


3 


2 


3 


3 


1 
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was marked. The methods used In the AIR analysis for this section are 
not known. 

Across all six years, states most commonly used a pretest-post test 
design, and they usually reported grade-equivalent gain scores for what- 
ever samples were used. The evaluation design matrix used in analyzing 
the FY 71, 72, 73, and 74 reports shows that the combination of pretest- 
posttest design with grade-equivalent gain scores was most commonly used 
within the majority of State Reports (see Table 4). Across all six years, 
states most frequently broke down scores by grade level. For all six 
years, scores were most frequently summarized by means, and measures of 
variability were rarely reported. However, the variety in reporting for- 
mats and methods of presentation is inadequately represented here. The 
length of time between pretest and posttest varied greatly within states. 
In some instances, gain scores over varying treatment intervals were com- 
bined per month; and in still other cases, scores were reported separately 
for each treatment interval. 

As can be seen in Table 5, many different tests were used by the 
states, although several states provided no information about what tests 
were used to obtain cognitive benefit data. Even when information on tests 
was provided, reviewers frequently were unable to determine exactly what 
test was being referred to, because reports often listed tests by giving 
incomplete titles. For example, when a report referred to the "Durrell," 
reviewers were unsure whether the Durrell Analysis of Reading Difficulty, 
the Durrell Listening Test, the Durrell Silent Speed Reading Test, the 
Durrell Oral Speed Reading Test, or the Durrell-Sullivan Test of Reading 
Capacity was meant. In FY 71, 72, 73, and 74, as in FY 69 and 70, when 
several tests were used within a state, the state sometimes pooled the 
results of various tests, sometimes reported results for only the more 
commonly used tests, or occasionally reported results separately for each 
test used. Across the years 1969 through 1973, the California Achieve- 
ment Test, the Gates MacGinitie Reading Tests, the Iowa Tests of Basic 
Skills, the Metropolitan Achievement Test, and the Stanford Achievement 
Test - or subtests - were most frequently reported. In FY 74, the SRA 
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TABLE 4 



Number of Reports which used Particular Types of 
Experimental Designs and Test Scares for iPy 71, 72, 73, and 74 



Pre-Post 



Post Only 



Fiscal Year 71 



Norm ref. Control Norm ref* Control 



Raw Scores 2 

Standard Scores 

Stanines 

Centiles , dec . , quart • 2 

Grade Equivalents 8 

Raw, Std., Stanine Gain 2 
Centile , dec . , quart . Shift 7 

Grade Equiv. Gain 36 

Residual Gain 1 
Criterion ref. 



Fiscal Year 72 



Raw Scores 
Standard Scores 
Stanines 

Centiles, )dec., quart. 
Grade Equivalents 
Raw, Std., Stanine Gain 
Centile, dec, quart. Shift 
Grade Equiv. Gain 
Residual Gain 
Criterion ref. 
Other not specified 



2 
1 

1 
4 
36 



Fiacal Year 73 



Raw Scores 
Standard Scores 
Stanines 

Centiles, dec, quart. 
Grade Equivalents 
Raw, Std., Stanine Gain 
Centile, dec, quart. Shift 
Grade Equiv. Gain 
Residual Gain 
Criterion ref. 
Other not specified 



2 
3 

2 

36 



Fiscal Year 74 



Raw Scores 
Standard Scores 
Stanines 

Centiles, dec, quart, 1 
Grade Equivalents 3 
Raw, Std., Stanine Gain 
Centile , dec . , quart . shift 2 
Grade Equiv. Gain 24 
Residual Gain 1 
Criterion ref. 1 
Other not specified 



* Only the most commonly used experimental design and test score 
were recorded for each report. 
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TABLE 5 



Frequency with which Various Standardized Tests 
were Cited in the State Title I Annual Evaluation Report! 
Presented by. Fiscal Year* 



Fiscal Year 



Standardized Test 


69 


70 


71 


72 


73 


74 


*-'c*xiiorma Acnieveinent iest 


•29 


20 


17 


25 


23 


7 


uomprenensive iest ox Basic Skills^ 






5 


10 


9 


2 


i^uLrexx ouxxivan iesii or Keadlng Capacity 


4 


4 


5 


5 


3 


2 


vjdLcfa nacoiniLie Keaaing iest 


16 


23 


29 


26 


29 


9 


vjray urax Keaaing iest 

V 


6 


2 


1 


3 


5 


2 


iowa lesLS ox caslc Slcllls 


21 


19 


16 


19 


19 


6 


Lee-Clark Readiness Test 


3 




1 


2 


4 




Metropolitan Achievement Test 


18 


20 


26 


29 


24 


7 


Nelson Silent Reading Test 






2 


5 


1 


1 


Pcabody Picture Vocabulary 


2 


2 


1 


6 


10 


6 


Sequential Test of Educational Progress 






1 


1 






SRA Achievement Test 


8 


8 


17 


16 


14 


8 


Stanford Achievement Test 


29 


23 


27 


26 


22 


7 


Wechsler Intelligence Scale 


2 




1 




1 




Wide Range Achievement Test 






6 


10 


17 


9 


Other 


6 


5 


38 


46 


55 


30 



As pretests, posttests, or both. 
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Achievement Test and the Wide Range Achievement Test were also frequently 
reported. In FY 71, 72, and 73, the tests that were reported most fre- 
quently were also reported by many states as being amonf. the most frequently 
used within the state (see Table 6.) This information was not compiled 
for FY 69 and 70, and the information for 1974 is too scanty to use in 
drawing generalizations. The tests shown in Table 5 as being used by 
several states in FY 71, 72, 73, and 74, and by none in FY 69 and 70, were 
added to the checklist during the current review and probably were not 
recorded when the FY 69 and 70 reports were jl*. viewed. 

Overall, a comparison of the results -of the content analysis across 
fiscal years suggests that there have not been any significant trends . 
Chi square tests performed on each of the three sections of the content 
analysis (target population, expenditures, and evaluation data) for FY 71, 
72, and 73 did not reveal any' significant differences in the frequency 
with which various ciata were reported. These analyses were not performed 
on FY 74 reports because of the small number of reports available, and 
because the analyses of earlier reports resulted in no significant differ- 
ences. 

The absence of trends in report;ing practices is particularly inter- 
esting because a Program Information Memorandum was issued by U.S.O.E. in 
April, 1971 rescinding all reporting guidelines which had previously been 
in effect. The" fact that the relaxation of requirements produced no 
noticeable changes in report content can only lead to speculation on 
a number of issues. One wonders, for example, how closely the guide- 
lines had been followed while in effect. If the states originally 
adopted only those portions of the guidelines with which they concurred, 
no changes would be expected. On the other hand, the system may have 
so much inertia that change, even in desired directions, rarely occurs. 
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TABLE 6 

Standardized Tests Used Most Commonly within States 
Presented by Fiscal Year 

Fiscal Yeal: 



Standarized Tes^t 71 72 73 74 

-> * . 

California Achievement Test 11 9 7 3 

Comprehensive Test of Basic Skills ^ 3 2 1-3 

Durrell-Sullivan Test of Reading Capacity 1 

Gates-MacGinitie Reading Test 17 16 16 5 

Gray Oral Reading Test .1 

Iowa Tests of Basic Skills 10 9 7 

Lee-Clark- Readiness Test 

Metropolitan Achievement Test 14 17 i^*"^ ^ 4 

Nelson Silent Reading Test 

Peabody Picture Vocabulary 1 . 

Sequential Test of Educational Progress . ^ 

SKA Achievement Test . 6 3 

i 

Stanford Achievement Test 17 n 13 4 

Wechsler' Intelligence Scale 1 

Wide Range Achievement Test 113 1 
* As pretests, posttests, or both. 
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Data Adequacy, Policy Relevance, and Readability Ratings 

In addition to the content analyses described in the previous chapter ^ 
the work statement for Contract OEC-0-74-9182 called for the rating of 
FY 71, 72, 73, and 74 State Title I Annual Evaluation Reports in terms of 
their readability and policy relevance, and the quality (representative- 
ness and validity) of the data they contained. Again, this activity 
was closely related to the previous AIR work (Wargo, et. al^, 1972) and 
made use, in fact, of the Scale of Adequacy and Validity of Reported 
Data developed by AIR. A copy of this scale is presented in Appendix 
B along with instructions for its use. Comparable scales for Readability 
^nd Policy Relevance (see Appendices C and D) were developed by the 
RMC project staff. 

Before discussing the RI4C-develapad scales, it is appropriate to 
summarize the features of the Data Adequacy Scale. The areas measured 
by the scale included the representativeness of the sample used to 
evaluate program success, the validity of the experimental design, 

J' 

and/ whether specific types of ' information were .presented concerning 
the target groups, program, and costs. The range of the total scale 
was 0 . 0 to 7 . 6 . 

The first portion of the scale is concerned with cognitive benefit 
information. Full credit was earned when achievement data were pre- 
sented for all students served or for a representative sample. 

The scale had anchor points corresponding to samples which ware 
judged "probably not "biased" and "probably biased." There was also 
an anchor point midway between these points which corresponded essentially 
to a "can't tell" situation. The number of points which could be ob- 
tained for' the "Representativeness of the Sample" was from 0.0 to 2.0. 

Once a sample rating was determined, it was adjusted according to 
the type and quality of the data presented. Points were subtracted 
for such deficiences as not presenting both pre- and posttest scores. 
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not reporting standard deviations, etc.. The adjusted rating was named 
the "Cognitive Benefit" portion of the scale (points could range from 
0.0 to 2.0). 

Target Group, Program, and Cost Information categories provided 
add-ons to the Cognitive Benefit scores, and the scores in each category 
were doubled if: (a) a score of 0.7 or more was obtained on the Cognitive 
Benefit dimension, and (b) the data contained in the categories were 
integrated with the cognitive data used to measure program outcomes. 
The Total Data Adequacy score was based upon adding the points- obtained 
on this portion of the scale to the Cognitive Benefit score. 

Development of the Policy Relevance and Readability Scales 

The Policy Relevance Scale was intended to reflect the current 
concerns of educational policy makers and legislators in accordance 
with the assumption that one of the main purposes served by the state 
reporting system is to provide these individuals with the information 
they need for the effective execution of their responsibilities. For 
this reason, interviews were set up and conducted by members of the 
RMC staff with key personnel selected by U.S.O.E. 

The Title I staff was interviewed in depth including individual 
sessions with the Director of Title I, Richard L. Fairley, and members 
of the Program Support Branch and Program Services Branch of. Title I; 
Paul Miller, William Lobosco, Joseph Vopelak, Velma James, and John 
Staehle. 

Interviews were conducted with other officials in the Office of 
Education as well. Duane Mattheis, the Acting Executive Deputy Com- 
missioner of Education; Robert Weatherford, Special Assistant to the 
Commissioner; Robert Wheeler, Deputy Commissioner for School Systems; 
and John Rodriquez, Associate Commissioner for Compensatory Education 
contributed their thoughts. 

Relevant policy question information was discussed with individuals 
directly concerned with the formulation of legislation involving Title 
I. Congressman Victor Veysey was interviewed as well as Christopher 
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Cross, House Minority Counsel to the Sub-committee on Education; Jean 
Froelicher, Counsel to the Senate Sub-committee on Education; and Charles 
Cooke, Deputy Assistant Secretary for Legislation. 

Alan Ginsberg of HEW/ASPE; Carl Wisler, Director of the Elementary 
and Secondary Programs Division (USOE/OPBE); and George Mayeske also 
of the Elementary and Secondary Programs Division added their insights 
concerning the policy issues to be considered. 

There was a general consensus among the individuals interviewed as 
to which policy issues were the most relevant. The emphasis in the 
discussions as well as in the new legislation was on the importance of 
being able accurately to describe and assess what is occurring at the 
level of the local Title I project. More specifically, information is 
desired concerning (1) the manner in which the needs of the children 
are originally assessed, (2) the number of children involved in a pro- 
ject, (3) the per-pupil expenditure of each project, and (4) evidence 
of project achievement benefits. In essence, the major concerns ex- 
pressed were centered on the necessity of knowing how Title I funds 
are being spent, how target groups have been defined, how needs of the 
groups have been assessed, and if the treatments have resulted in sig- 
nificant educational gains. Information in this form, presumably, could 
then be aggregated to reveal state- and nationwide pictures of- the impact 
of Title I. 

The number of non-public school children as well as the number 
of public school children participating in Title I was an issue of 
interest to many of those interviewed. Further, there was a desire to 
know if the services received by these two groups are comparable in 
terms of meeting the needs of these children, not simply on a cost per 
pupil basis. 

Four other issues were mentioned less frequently and would appear 
to be of secondary interest to those interviewed. First, concern was 
expressed about the need to have relevant information on the Parent 
Advisory Councils, i.e., that they do exist, their function, their 
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membership, etc.. v^Second, the results of a study mentioned by one inter- 
viewee indicated that a positive correlation exists between the number 
of hours spent in the classroom and the amount of achievement gain. It 
was therefore suggested that the amount of instructional time and the 
nature of the time (one-to-one, small group, regular classroom) would 
be of interest in developing policy. Third, in trying to decide if 
funding and program efforts should be concentrated at the early grade 
levels, the elementary grades, or the secondary grade levels in order 
to achieve maximum benefits, it was mentioned that any information per- 
taining to the most effective *'time of intervention" would be of interest. 
Fourth, data concerning the availability of other state and federal 
compensatory education funds within the states would be helpful in 
deciding whether Title I funds act as an incentive or a hindrance to 
the d evelopment of other compensatory education programs. 

After conclusion of the interviews,*^ the issues were listed .and 
then each issue was appropriately weighted to reflect its relative 
importance to the policy makers interviewed. The result of this pro- 
cedure was the creation of the Policy Relevance Scale which is attached 
as Appendix D, The State Title I Reports for the fiscal years 1971, 1972, 
1973, and 1974 were subsequently rated with this scale, and the results are 
presented below. 

The other scale developed by the RMC staff was the Readability Scale. 
It appeared that this type of scale was necessary to determine whether the 
reports adhered to basic standards related to clear technical descriptions 
and were organized in a coherent manner. Eight format and stylistic dimen- 
sions were identified through review of standard texts on the subject, and 
point assignments within each dimension were worked out through group con- 



Additional interviews were conducted after development of the Policy 
Relevance Scale. The interviewees were Congressman Albert Quie; 
John Evans, Assistant Commissioner for Planning, Budgeting and Evalu- 
ation; Al Alford, Assistant Commissioner for Legislation; Jack Jennings, 
House Majority Counsel to the Subcommittee on Education; and Roy Millen^ 
Counsel to the Senate Subcommittee on Education. In general, the 
comments of these interviewees were in close agreement with those 
described above. 
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census. Six different versions of the scale were developed before it was 
generally felt that ambiguities had been removed and adequate between- 
rater agreement could be achieved. 

Interrater Reliability • 

. The rating of reports was accomplished by four members of the 
project staff^ Each rater worked with several reports and discussed 
questions with other raters before the formal rating process began. At 
that time a random sample of 30 reports was selected for the purpose of 
assessing interrater reliability. 

In the most commonly encountered rating situation, two or more 
raters rate a set of entities. The extent of their agreement is typically 
measured in terms of a coefficient of correlation and, if it is adequately 
high, ratings^ for each entity are subsequently determined by pooling the 
values assigned by the individual raters. This procedure is entirely 
suitable for the situation where all raters rate all entities. In the 
case of the State Reports, however, it was clear from both the number 
of reports to be rated and the length of time required to rate each re- 
port that the rating task would have to be shared. It would not be 
possible for any rater to rate all reports within the existing time 
constraints. 

Under these conditions, the reliability issue becomes more complex 
and it is necessary to demonstrate not only agreement with respect to 
the relative ratings assigned to different entities but with respect 
to the absolute values of these ratings as well. This type of agreement 
cannot be measured using correlation techniques. An analysis of variance 
technique was therefore adopted. 

The first analysis of variance was computed after 12 reports had 
been independently rated by each of the four raters. The results. are 
presented in Table 7 which shows statistically significant differences 
among raters, among states, and among scales; and significant rater-by- 
scales and states-by-scales interactions. 

The difference among states was to be expected and siiaply reflected 
real differences in the quality of different reports. Similarly, the 
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TABLE 7 



Analysis of Variance Performed on Data from Twelve States 
(Ratings by Individuals) 



Source 


SS 


df 


MS 


F 


Raters (A) 


13.37 


3 


4.46 


5.79* 


States (B) 


162.16 


11 


14.74 


19 . 14* 


Scales (C) 


1450.05 






627 . 73* 


A X B 


29.76 


33 


0.90 


1.17 


A X C 


41.53 


9 


4.61 


5.99* 


B X C 


107.60 


33 


3.26 


4.23* 


A X B X C 


76.10 


99 


0.77 




* p < .01 










among-scales 


difference simply 


reflected 


the fact that 


the scales were 



scored differently and had different ranges of possible scores. The 
among-raters difference and the raters-by-scales interaction, on the 
other hand, signified that different raters did, in fact, employ different 
rating strategies or criteria. 

Visual inspection of the rating data revealed that there appeared to 
...be two distinct patterns of ratings, each reflected in the values as- 
signed b)K^wo vf the raters. Thus, it appeared that two similar "teams" 
could be formed by pairing the dissimilar raters. The overall difference 
between pairs was essentially zero and a Chi Square analysis showed no 
significant deviations from expectations in the matrix of rater-by- 
scale values. It appeared that, if this pattern held up, it would be 
possible to allow pairs of raters to rate different reports without 
compromising the reliability of the ratings. A decision was therefore 
made to proceed with the remaining 18 reports in the' selected sample 
to determine whether the pairing strategy would continue to be effective. 

Table 8 presents the results of the analysis done on the 18 reports 
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TABLE 8 



Analysis of Variance Performed on Data from Eighteen States 
(Ratings by Individuals) 



Source 


SS 


df 


MS 


F 


Raters (A) 


4. 35 


3 


1.45 


1.88 


States (B) 


318.69 


17 


18.75 


24.35* 


Scales (C) 


1585.03 


3 


528.34 


686.16* 


A X B 


72.53 


51 


1,42 


1.84* 


A X C 


75.56 


9 


8.40 


10.91* 


B X C 


252.76 


51 


4.96 


6.44* 


A X B X C 


118.57 


153 


0.77 





* p < .01 



which showed that there was no significant difference between raters 
across the four scales- However, as was the case with the 12 reports, 
the raters-by-scales and states-by-scales interactions were significant. 
Unfortunately, the X'^ tests again revealed that the raters-by-scales 
interaction was significant even after the raters* scores were paired in 
the same manner as was previously described- Pairing the raters did, 
however, as in the twelve-state analysis, reduce the size of the inter- 
action substantially- It also eliminated the raters-by-states interaction. 

The between-pairs reliability coefficients were calculated for all 
30 reports for each of the four scales (the Cognitive Benefit portion 
of the Data Adequacy Scale was considered separately)- The following re- 
sults were obtained, ' 

Cognitive Benefit = ,70 
Data Adequacy = -82 

Policy Relevance = .76 
Readability = .82 

These reliability results were considered marginal and, in a further 
attempt to increase reliability, the pairs were 'asked to re--rate those 



reports on which substantial disagreement was observed. The members of 
each pair were asked to discuss their ratings with one another in order 
to reach more agreement with respect to certain states. However, no dis- 
cussions took place between the pairs. 

The re-ratings of the reports produced small but meaningful increases 
in reliability, and the following between-pairs reliabilities were obtained 



Cognitive Benefit = .74 

Data Adequacy = .87 

Policy Relevance = -83 

Readability = .89 



The "paired/' "discussed" ratings for all 30 states were also sub- 
jected to an analysis of variance an'd the results are presented in Table 9. 



TABLE 9 

Analysis of Variance Performed on Data from Thirty States 

(Ratings by Pairs) 

Source SS ' df MS F 

Pairs (A) 0.14 1 0.14 0.14 

States (B) 1057.79 29 ' 36.48 37.61* 

Scales (C) 6034.44 3 2011.48 2037.69* 

A X B 62.43 29 2.15 2.22* 

A X C 27.69 3 9.23 9.52* 

B X C 786.68 87 9.04 9.32* 

A X B X C 84.09 87 0.97 



* p < .01 

Examination of the pairs-by-scales interaction revealed that the 
differences between pairs were again significant for the Data Adequacy 
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and Readability Scales. Because these differences were also found to be 
remarkably consistent across the 12 and 18 report samples, it was concluded 
that this particular interaction effect could be effectively countered by . 
using appropriate weightings. Consequently, each of the ratings of the 
"low" team was increased by an amount equal to the mean difference between 
the pairs pn the 30-report reliability sample. Unfortunately, no similar 
solution came to, mind for dealing the with rater-by-states interaction and 
this source of variance had to be considered error. 

Further avenues for improving inter-rater reliability could not be 
identified and the decision was made to proceed using the approach just 
described. Each report was rated by one team whose two members discussed 
their independently made ratings wherever discrepancies were found. The 
two ratings of each report were then averaged and those of the lower-rating 
team were weighted to adjust for the established difference between teams. 

Procedures Followed for Rating All of the State Reports 

The 157 state reports not included in the reliability sample were 
assigned to pairs randomly by state in an attempt to maximize the com- 
parability of the analyses of trends from year to year. Where one or two 
of a state's reports were included in the reliability sample (and thus had 
been rated by both teams) , the ratings of the taam assigned the remaining 
reports of that state were retained and those of the other pair were dis- 
carded. There were two instances in which all the reports from a state 
were included in the reliability sample. In these cases, the pair whose 
ratings were retained was decided by flipping a coin. This approach 
appeared to offer the most reasonable method for achieving comparability 
between the ratings of reports in the reliability sample and those not in it. 

After the ratings were completed, discussed by the pairs of raters, 
and possibly revised based on the discussion, the mean rating of each pair 
was computed for each scale. Then, weights were used to adjust for the 
differences between pairs observed on the rating of reports in the relia- 
bility sample. The low scoring pair's ratings were increased by 0.7 on 
the Total Data Adequacy Scale and by 1.1 on the Readability Scale for each 
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state. The weighting procedure was not uged on the Cognitive Benefit and 
Policy Relevance Scales because the pairs were not significantly different 
on these scales. The weighted, average ratings were then normalized to 
eliminate the skewness (piling up of ratings at the low end of the scale) 
reflected in the original sc ores and standardized to form an equal- interval 
scale with a mean of five and a standard deviation of two. This trans- 
formation was identical to that performed by AIR and was intended to enable 
comparison of our results with those obtained by Wargo and his colleagues 
in the cognitive benefit and data adequacy areas. Unfortunately, ratings 
by states are not included in the AIR report and it has not been possible 
to retrieve them from the AIR "archives." 

Overall Consistency of Reporting Practices 

To assess the overall consistency of reporting practices followed by 
the states, correlations were calculated between years separately for 
the cognitive benefit, data adequacy, policy relevance, and readability 
ratings. These correlations are presented in Table 10. Although most 
of the correlations were statistically significant, the size of the 
relationships indicated that there was considerable variation in the 
quality of individual State Reports from year to year. The correlations 
between 1972 and 1973 indicated that across all scales the ratings for 
those years were more alike than those of any other p.air of years. The 
highest correlations were found between 1972 and 1973 on the readability 
scale and between 1973 and 1974 on the cognitive benefit and total data 
scales, but even these figures reflected substantial variation between 
years. 

The increases in the correlations on the cognitive benefit and total 
data scales from 72-73 and 73-74 as compared to 71-72 suggest that the 
states became more consistent in their reporting procedures. Interestingly 
enough, the AIR study (Wargo, et al. , 1972) found even lower correlations 
than those listed in Table 10. However, the correlations between 73-74 on 
the policy relevance and readability scales are quite low. It is possible 
that these low correlations are due in part to the fact that only 30 
FY 74 reports were available at the time of the analysis. To determine 
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whether the variations in reporting quality were random or reflected 
systematic trends from year to year, jt tests were calculated between 
years using the same rating data. The obtained t^ ratios are also 
presented in Table 10. 

The negative _t values in Table 10 indicate higher mean ratings for 
the more recent of the two years being compared than for the earlier year. 
Since all of the _ts were negative, there appears to be a trend toward 
improved reporting practices. Thirteen of the Jts were statistically 
significant, most of them reflecting comparisons of 1974 with the preced- 
ing years. Again, this may be due to the small number of 74 reports avail- 
able rather than to a systematic trend. 

Cognitive Benefit Scale . The cognitive benefit portion of the 
Data Adequacy Scale was designed to measure whether adequate sampling 
procedures were followed in selecting the children who were tested, and 
the number of points on the scale ranged from 0.0 to 2.0. The median 
ratings obtained for the years 1971, 72, 73, and 74 were 0.7, 0.8, 0.6, 
and 0.7 respectively. Thus, a\ost of the reports were judged inadequate 
with respect to sampling and the majority of the ratings fell into the 
"Biased", "Probably Biased", and "Possibly Biased" categories (See Table ' 
11). However, it must be emphasized that no guidelines for selecting 
sample3 had been provided to the states, nor had it been suggested to them 
that samples should be representative . Clearly, however, guidance^^lth..^ * 
respect to sampling will be required if future reports are to provide a 
valid indication of Title I impact. 

Table 11 shows that the major changes which occurred between 1971 
and 1974 were in the "Probably Biased" and "Probably Representative" cate- 
gories, as indicated by a 22-point decrease in the first category and a 
15-point increase in the second. In addition, 92, 85, 83, and 74 percent 
respectively of the ratings for these four years were in categories which 
ranged from "No Cognitive Benefit Data" to "Possibly Biased." Only 8, 
15, 17, and 26 percent of the states appear to have used representative 
samples . 

Table 12 compares the percentage of states which reported on samples 
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TABLE 11 



Years 



1971 
1972 
1973 
1974 



Percentages and Median .Ratings of Reports in Each Cognitive 

Benefit Category- 



No Cognitive Probably Possibly Probably 

Benefit Data Biased Biased -Biased Representative Representative 
0.0 0.1-0.4 0.5-0.9 1.0-X.4 1.5-1.9 2.0 



2 
12 
8 
7 



Mdn 



y Mdn % Mdn 



19 0.3 45 0.6 26 1.0 8 

13 0.3 37 0.7 23 1.1 15 

8 0.3 42 0.7 25 1.0 11 

3 0.1 23 0.7 40 1.1 23 



Mdn 



1.7 
1.7 
1.7 
1.7 



0 
0 
6 
3 



TABLE 12 



Percentage of States whicli Selected 
eitlier Biased or Representative Samples 



No Cognitive Certainly to Probably to Possibly Biased 

Years Benefit Data Probably Biased Probably Biased to Representative 



1969 and 70* 
(n-91) 

1971-1974 
(n-187) 



40 



17 



25 



46 



26 



30 



* From (Wargo, 1972) 
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ranging from "Certainly Biased" to "Representative" dn 1969 and 70 (from 
Wargo, et al., 1972) with those from 1-971^ 72, 73, and 74. Although the pe 
ce^ntage shifts in Table 12 indicate that the 1971-74 reports had relatively 
fewer "Certainly to Probably Biased" samples than did the 1969-70 reports, 
the percentage of^ repprts which were based upoii- representative samples 
did not sh6v> a comparable improvement. It appeared that there was some 
improvement in the more recent samples but the improved level was still 
far below minimally acceptable .standards . . . 

Clearly, consistent and representative sampling procedures were not 
used by most of the states. Whether this deficiency can be attributed 
to the absence of specific guidelines concerning valid sampling procedures 
still must be determined.^ It seems ^ikely, however, that there will be..'* 
difficulties associated with gathering adequate evaluation samples which 
will not be removed by the simple prov^ision of guidance. 

Policy Relevance Scale . The median ratipgs and ranges for the Policy 
Relevance Scale "Total Score^ are presented in Table 13. Table 14 shows 
the frequency distributions of scores for the five issues felt by the ed- 
ucational policy makers interviewed to ,be the most significant. Clearly, 
most of the states obtained low sc9res on these issues. It is interesting 
to note that the highest ratings occurred in the Achievement Benefit area 
with the Resource Allocation area second. The Public/Nonpublic participa- 
tion issue was very inadequately covered while most states: also received 
ratings in the lowest range on the Parent Advisory Council and Needs 
Assessment issues. 

It can be seen that the ratings were quite low with even the highest 
rated report earning slightly over half of the maximum possible score. • 
Again, however, it should be emphasized that these ratings were made in 
a post hoc manner, i.e., the scale was developed and used to rate reports 
which had been written without any relevant guidelines. For this reason, 
the ratings in no way reflect either the willingness or the ability of the 
states to address the issues and thus, comparisons among states are not 
meaningful. „ ' f . 
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TABLE 13 



Median and Range of Ratings on the Policy Relevance Scale 
(Raw Scores) Presented by Fiscal Year 



1971 1972 1973 1974 



Median 5.2 4.9 5.5 6.4 

Range 1.6-8.5 0.8-8.8 0.5-10.4 2.5-9.0 
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TABLE 14 



Percentage of Reports which Obtained Ratings 
within each Policy Issue 



Rating Categories 
Policy ] • _ • 

Issues 0.0-0.9 1.0-1.9 2.0-2.9 3.0-3.9 4..0 



Achievement 
Benefit In- 
formation 



1971 11 42 43 4 0 

1972 17 35 46 0 . <> 

1973 11 29 58 2 0 

1974 7 10 . 70 10 3 



0 = No achievement data^ 

4 = Achievement test data; pre, post, and gain by project and aggre- 
gated across state^ 

Needs 

Assessment 



1971 


79 


11 


8 


2 


0 


1972 \ 


75 


19 


6 


0 


0 


1973 ) 


84 


6 


6 


4 


0 


1974 (1 


57 


30 


10 


3 


0 



0 = No needs assessment data 

4 = Project level descriptions of needs assessment procedures and 
results plus statewide summary 

Resource 
Allocation 



1971 38 25 38 GO 

1972 31 23 42 4 0 

1973 31 23 42 4 0 

1974 33 13 53 0 0 



0 - No data on expenditures 

3 = Project level breakdown of expenditures by service area and 
statewide aggregation 
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TABLE 14 (cont.) 



Policy Rating Categories 

Issues 0.0-0.9 1.0-1.9 2.0-2.9 3.0-3.9 4.0 



Publlc/Nonpubllc 

1971 94 6 0 0 0 

1972 94 6 0 0 0 

1973 94 6 0 0 0 

1974 93 7 0 0 0 



0 = No breakdown 

2 = Breakdown of expenditures and number of participants by county 
plus statewide aggregation 

Parent Advisory 



Councils 

1971 85 15 0 0 0 

1972 81 19 0 0 0 

1973 77 23 0 0 0 

1974 63 37 0 0 0 



0 = No Information 

2 = School or district level descriptions of PAG membership compo- 
sition, activities, and achievements plus statewide summary 



1. Number of points for lowest rating. 

2. Number of points for hlgest rating. 



48 

r" ^ 
0 0 



Readability Scale . The analysis of the readability scale was con- 
fined to the total score for each state. Since there was a high correla- 
tion among the rating elements, a breakdown by elements did not appear 
to be useful. The total readability score seemed to reflect the overall 
quality of the reports as well as frequency counts of ratings on individual 
elements- A high total score on this scale usually indicated high ratings 
in categories concerned with the abstract, organization, integration, writing 
style, technical presentation, tables and figures, summary, and copying. It 
should be noted that the readability ratings were substantially higher than 
the ratings on the other scales. This fact would seem to suggest that, where 
guidelines are generally known, reporting quality is high. 

The data shown in Table 15 reveal that a slight increase in the 
readability ratings occurred between 1972 and 1973 and between 1973 and 1974. 
The increase between 1973 and 1974 does not appear to be caused by selective 
attrition factors, since the states for which '*^/4 reports were unavailable 
included almost an equal number of low and high 1973 ratings. 



TABLE 15 

Median and Range of Ratings on the Readability 
Scale Presented by Fiscal Year 

1971 1972 1973 1974 



Median , 7.2 7.2 8-0 8.7 

Range 3 . 2-10 - 7 2 -0-11 .0 4 - 1-10 - 2 5 - 8-10 - 5 
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Conclusions' and Recommendations 

This chapter has described the analyses which were undertaken during 
Phase I of the four -phase study as well as the findings which resulted 
from them. While the findings may be interpreted in several different 
contexts, it should be kept in mind that the primary purpose of the Phase I 
effort was to assess the current state-of-the-art in Title I evaluation 
reporting for the purpose of generating reasonable recommendations for 
improved evaluation and reporting practices . 

The standards for evaluating the reports were (a) a Policy Relevance 
Scale developed on the basis of interviews with carefully selected, fed- 
eral-level, educational policy makers, (b) a Data Adequacy Scale developed 
to reflect technical considerations affecting the meaningf ulness and inter- 
pretability of statistical information contained in the reports, and (c) 
a Readability Scale designed to reflect well-accepted principles of exposi- 
tory presentation, A content analysis was also undertaken in Phase I 
and provides an additional basis for evaluation when actual content is 
compared against the expressed desires of the policy makers who were 
interviewed. 

It was clear, long before the formal analysis was completed, that 
the typical State. Title I Annual Evaluation Report f^lls far short of the 
ideal, A substantial number of the reports which were examined contained 
no data which could be aggregated, and the sum total of the reports could 
not be used to synthesize a national-level assessment of Title I effective- 
ness. These findings, however, came as no surprise. 

Consideration of the quality of the average report also provided 
little insight as to what kind and how much improvement it might be 
possible to effect through education, guidelines, or even sanctions. The 
formal analyses described in previous sections of this report were of 
little value in seeking answers to these questions. Some insights , how- 
ever, could be obtained from individual reports, A few states provided 
reasonably representative and interpretable statistical data for at least 
one of the three years. Also, all of the policy issues were addressed 
completely by one or more of the reports. Arid many of the reports were 
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quite well written. These observations made it clear that the standards 
against which the reports were assessed are not unattainable individually — 
although in aggregate they may be. 



In general, it appeared that those states whose reports contained 
the best statistical data were those which had standardized evaluation 
and. reporting practices on a statewide basis. Where this situation 
prevails, of course, it is possible for the State Department of Education 
to aggregate the reports they receive from the LEAs with a minimum of 
difficulty. Where states provide no guidelines, they are often faced 
with the impossible task of aggregating such incompatible information 
as average monthly gains reported by some LEAs with quartile shifts 
reported by others. 

Clearly it is desirable to allow the LEAs some flexibility in 
the choice of their own evaluation and reporting practices. Further- 
^ore, whatever standardized practices are adopted must be acceptable 
and useful to the LEAs. Unless some standardization is effected, how- 
ever, there is little hope that State Reports will ever be useful 
for obtaining a national-level assessment of Title I impact . The stan- 
dardization must, of course, be accomplished across as well as within 
states, and this need may constitute the most difficult obstacle to be 
overcome in achieving the study's objectives. 

It follows from the preceding paragraphs that there appears to be 
no theoretical barr ier preventing State Title I Annual Evaluation Reports 
from inc luding coverage of the desired content areas and presenting the 
desired informatio n in a standardized format which would enable aggrega- 
tion acr oss states . The problems with achieving these goals stem from 
practical rather than theoretical considerations* and the analyses con- 
ducted to date .are of little help in trying to determine how closely 
the objectives can be approximated. Compromises will be required, but 
it seems possible to achieve a suitable tradeoff between the amount of 
information presented and the quality of the data so that whatever is 
contained in future reports will be both meaningful within states 
and suitable for aggregation across states. 
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One problem area where no compromise appears workable is that of 
assessing and reporting cognitive achievement benefits. As shown by 
the content analysis, the great majority of cognitive benefit assess- 
ments have been based on norm-referenced comparisons and reported in 
terms of grade-equivalent scores or grade-equivalent gains. These 
characteristics probably invalidate all such evaluations. Work done 
by RMC in conjunction with another project (Tallmadge & Horst, 1974; 
Horst, Tallmadge, & Wood, 1975) has demonstrated convincingly that 
grade-equivalent scores cannot provide valid _gain indices when used 
in norm-referenced evaluation models and that the valid use of such 
models imposes stringent requirements on testing times and the use 
of normative data which are generally not known or recognized. SEAs 
and LEAs mist be taught the pitfalls associated with the models they 
are currently using, and must ' e convinced of the necessity of using 
valid designs such as those presented in Horst» et al. (1974) . . 
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III. PHASE II 



Summary of Phase II Tasks 

As the first task of Phase II, each State Report for FY 71, 72, 73, * 
and 74 was re-read by one of the staff members responsible for the Phase I 
ratings, and the contents were described in the form of a critical summary . 
Each critical summary included a description of the following topics: 
participation, expenditures, cognitive impact, needs assessment, and 
Parent Advisory Councils, as well as the general readability of the report. 
If the report contained no information on a topic, this fact was noted in 
the summary. The critical summaries also included the standardized scores 
each report received on the Cognitive Benefit, Data Adequacy, Policy Rele- 
vance, and Readability Scales completed in Phase I. 

While these ratings provided a quantified numerical evaluation of 
the reports, the critical summaries provided a more detailed narrative 
description and assessment of the same material. In essence, the critical 
summaries pointed out the strengths and weaknesses of each report. They 
also illustrated the review process each report underwent during the ratings 
in Phase I. An example of a typical critical summary and a more detailed 
explanation of their exact content are given in the next chapter. 

The second and third tasks of Phase II involved the extraction and 
aggregation of data from the state reports that related to resource 
Allocation, participation, and cognitive achievement. Information relevant 
to needs assessment and Parent Advisory Councils was also excerpted. Not 
all states reported information on each of these categories, and those that 
did often employed different summarizing formats precluding the aggregation 
of mtore than a few states at a time. The aggregates of useable data are 
clearly not representative of the entire nation; however , in most cases, 
the data are representative of the states reporting them. 

It should not be assumed that reports containing data which could be 
aggregated were superior to reports which had adopted other reporting for- 
mats. In fact, the existence of common formats appeared to be more coinci- 
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dental than the result of any conscious effort either to optimize the use- 
fulness of the data presented or to achieve comparability. 

A larger number of states contributed comparable data to the category 
of participation than to resource allocation. The greatest difficulty in 
finding comparable data across the states was in the area of program impact. 
Before aggregation was possible, a state's impact data had to be based on 
a representative sample of children and had to share the format of another 

state. 

A description of the reliability and validity standards required of 
the data in each category and the methodology involved in making the 
aggregations are discussed in later chapters of the report along with 
the aggregations themselves. 

The final Phase II task was the analysis of within-year and across- 
year program trends . As far as possible, the aggregations in the present 
study were designed to permit comparison with those in Wargo, Tallmadge, 
Michaels, Lipe, and Morris (1972) and thus enable the trend analysis to 
encompass FY 69-70 as well as FY 71-74. In some cases, however, the 
methodology used by Wargo, et al. (1972) was incompatible with the objec- 
tives of the present study, and discrepancies occurred that limited the 
number of generalizations that could be made across the six years. Spe- 
cific differences in procedures are discussed at relevant places in later 
chapters. 

Each trend analysis, regardless of category, was also limited in that 
it was based on a non-representative sample of states. It could not be 
concluded that the observed trends existed on a national, or even a regional 
level. Despite these drawbacks, some trends within each category were 
identified and they are discussed in the report. 
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Critical Summaries 



\ 



During the second phase of the project, a critical summary of each 
State Title I Annual Evaluation Report was prepared by one of the staff 
members responsible for rating the report during Phase I, These sum- 
maries briefly discussed the reports' readability and described the 
contents of the reports related to issues of target population, resource 
allocation, cognitive impact, needs assessment, nonpublic participation, 
and Parent Advisory Councils. Topics covered by the three rating scales 
(Data Adequacy, Policy Relevance, and Readability) used earlier in the 
project were summiirized, as were aspects of data presentations which 
were not included in the rating scales but which were relevant to the ^ 
aggregation of data across states. 

Preparation of the critical summaries required a closer inspection 
of the reports than had been necessary during Phase I of the project. 
In each report, all data on each topic were examined and compared in 
an effort to determine their reliability and validity. For example, 
if several different breakdowns of target population or resource alloca- 
tion data resulted in different totals, this contradiction was reported 
in the critical summary and an attempt was made to discover an explanation 
for the discrepancy. Wherever possible, the critical summaries also 
discriminated between duplicated and unduplicated counts of participants 
and between reports of allocations and of actual expenditures. Patterns 
of reporting participation and resource allocation data, and additional 
information which could be computed from data in the reports were also 
discussed. 

Cognitive impact data in the State Reports were described in terms 
of the sample of students for whom data were presented and the "represent- 
ativeness" rating this sample received on the Data Adequacy scale. This 
rating was considered particularly important since unrepresentative 
samples could not be included in aggregations across states. Samples 
which received ratings below the level of "possibly biased" were not 
considered suitable for inclusion in an aggregation. Impact data were 
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also examined for deficiencies which were not covered by the rating scale, 
such as the inclusion of different students in pretest and posttest data, 
or the mixing of summer term data (having very short pre- to posttest in- ■ 
tervals) with regular term data. The methods of presenting impact data 
and the amount of explanatory material accompanying the data were also con- 
sidered in writing this section of the summary. The types of scores re- 
ported, breakdowns used, and any analyses presented were discussed, as well 
as whether or not tests and testing times were reported. 

The critical summaries also described any information the reports pre- 
sented on the methods and outcomes of needs assessments, on the membership and 
activities of Parent Advisory Councils, and on nonpublic school participation in 
Title I. Additional policy-relevant topics which were discussed briefly if they 
appeared in the reports were comparability. Title I instructional time per pupil, 
time of Title I intervention, integration of Title I with other compensatory 
programs, and the administrative activities of the SEA. 

The final topic of the critical summary was the readability of the State 
Reports. In this section the reviewers attempted to point out any difficulties 
they encountered in reading the report or in extracting data from it, and brie.fly 
to explain any factors which significantly influenced the report *s score on the 
Readability scale. The critical summaries also included, at the beginning, each 
report's normalized, standardized score on each of four scales: Cognitive Bene- 
fit, Total Data Adequacy, Policy Relevance, and Readability. 

Each State Title I Coordinator received a copy of the critical sum- 
maries of his or her state's reports at the beginning of Phase III of this pro- 
ject. The summaries were accompanied by an explanatory letter which also 
prepared these officials for the telephone calls they were to receive from 
project staff during Phase III. 

A sample critical summary follows on the next page. This is not a sum- 
mary of an actual State Report, but is a compilation of ratings and descriptive 
sections taken from the critical summaries of several different reports. It 
illustrates the basic intent of all the Critical Summaries, which is to describe 
specific characteristics of the Title I evaluation report for that year, to 
state the report's problems in such a way that suggested improvements are obvious, 
and to mention the report's strong points in such a way that the practices are 
incorporated into future reports. 
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Critical Summary 



CALI^SIPPI 
FY 72 

Ratings 

Cognitive Benefit: 3.6 out of a possible 10 
Total Data: 5.8 out of a possible 10 
Policy Relevance: 6.3 out of a possible 10 
Readability: 7-6 out of a possible 10 

Target population 

The report provides an unduplicated statewide count of participating 
students in a breakdown of public/nonpublic by regular year/summer/af ter 
ho urs . Figures in this breakdown can be summed to produce what is pre- 
sumably an unduplicated total. Breakdowns by instructional program, 
grade level, and county are also presented,. Ten exemplary projects 
are also described, and numbers of students participating in each of 
these projects are given. No information is provided on numbers of 
Title I eligible students who were not served. \ 

Expenditures 

The total amounts of money granted to the LEA's for each year since 
1965-1966 are reported. Per-pupil expenditures are presented for some 
selected projects in language arts. 

Impact data 

Impact data are reported for 25 reading projects which represent- a 15% 
sample. The method used for sample selection is not clear i however , the 
report states that a review of the sample revealed a wide and represent- 
ative range of project objectives, sizes, locations, and expenditures. 
The sample was rated "probably to possibly biased," and the data were not 
aggregated. Results are presented by project, and include information 
on project cost, number of students, cost per pupil, and number of 
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students scoring in each quartile on the pre- and posttest. The numbers, 
of students taking the pre- and post tests do not match, however, and this 
indicates that systematic biases may have been introduced into the data. 
There is no information on tests or test intervals. 

Needs assessment 

The report contains a discussion of needs assessed by the statewide 
testing program. No information is reported on local assessments. 

Public /Nonpublic participation % 

Nonpublic participation and total and per-pupil expenditures for non- 
public school participants are reported by grade band and school session. 

"\ 

Parent Advisory Councils 

The membership composition of PAC's and PAC activities are reported in 
a statewide summary. 

Readability 

The report is well organized and written in a clear and unambiguous manner. 
Most tables are easily readable. Statements summarizing the major find- 
ings included at the end of the report provide a good overview for the 
reader . 
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Participation 

State Title I Annual^ Evaluation Reports presented participation - 
data which were more comprehensive and reliable than data on resource 
allocation and cognitive impact. The major problem associated with the 
aggregation of participation data was in dealing with duplicated versus 
unduplicated counts. .-^Unduplicated counts of Title I participants are 
desirable for total state participation and grade level breakdowns, 
while an accurate picture of participation by program requires a dup- 
licated count. Some State Reports gave duplicated counts in both 
instances. In a number of State Reports, the figures were not labeled 
duplicated or unduplicated, and it was difficult to determine how they . 
were derived. In Wargo, et al., (1972) there was no distinctipn made 
between duplicated and unduplicated counts for the FY 69 and FY 7^0 par- - 
ticipation data. Here, there is an indication for each breakdown 
whether duplicated or unduplicated counts were used. Because of this 
difference in aggregation procedures, conclusions that can be drawn 
from participation data across the years 1969 through 1974 are limited. 

The State Reports contain participation data that are not included 
in this report since only those data that were presented in a format shared 
by several states could be aggregated.' Data were not aggregated from State 
Reports that contained information on regular year participation only. 

Table 16 presents the total Title I participation by state for 
fiscal years 1969 through 1974. The 1969 and 1970 figures, which have 
been taken from the Wargo, et al., 1972 report, include migratory and 
institutionalized participants, and are usually based upon duplicated 
counts of children. The figures for 1971 through 1974. exclude migrants 
and institutionalized children, an(J are based on- counts clearly labeled 
unduplicated .or counts which could be presumed unduplicated. A footnote 
indicates each of those cases in which it was not clear from the State 
Report whether the count was duplicated or unduplicated. 

For all six fiscal years, there are large differences in par- 
ticipation among states. For FY 70 and 71, Nevada had the smallest . 
number of Title I participants, while New York had the largest. In 
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TABLE 16 



Pupil Participation in Title I Presented by State and* 
Fiscal Year 



State or 



Territory 


FY 69 


FY 70 


FY 71 


FY 72 


FY 73 


FY 74 


Alabama 


664,271 


777,634 


DNA 


DNA 


DNA 


DNA 


Alaska 


12,463 


16,777 


17,321^,3 


3,106^ 


4,400^ 


RNA 


Arizona 


103,294 


61,930 


49,744 


44,272 


37,983 


36,220 


Arkansas 


149,616 


154,524 


158,223 


124,599 


96,317 


72,3121.** 


California 


251, 3U 


223,723 


258,221 


314,281 


343,627 


596,921 


Colorado 


53,355 


38,600 


37,840^ 


39,513 


46,819 


42,784^ ■ 


Connecticut 


33,579 


41,505 


41,207 


41,622 


39,610 


40,654 


Delaware 


10,313 


RNA 


6,685 


6,787 


6,988 


7,444 


Wash. D. C. 


RNA 


DNA 


DNA 


DNA 
97,713^ 


DNA 


DNA 


Florida 


197,523 


107,496 


46,517 


DNA 


82,127 


Georgia 


204,024 


RNA 


182,148 


15^,904 


131,107 


115,771 


Hawaii 


8,891 


7,874 


DNA 


DNA 


7,966^ 


RNA 


Idaho 


34,742 


38,592 


44,965 


33,200 


17,226 


20,396 


Illinois 


201,533 


RNA 


242, 788^ 


148,844 


138,513 


134,711 


Indiana 


154,493 


123,847 


120,438 


122,826 


136,468 


119,507** 


Iowa 


95,547 


114,084 


85,007^ 


78,330^ 


67,905 


RNA 


Kansas 


64,481 


6l9,299 


6S;,189 


51,643 


41,336 


RNA 


Kentucky 


DNA 


DNA 


DNA 


RNA 


238,122 


DNA 


Louisiana . 


228,030 ' 


157,949 


163,283 


145,988 


120,302 


RNA 


Maine 


38,115 


33,459 


DNA 


31,316 


RNA 


27,727 


Maryland 


RNA 


56,781 


87,385 


68,944 


DNA' 


RNA 


Massachusetts 


103,071 


. 69,824 


68,985 


75,886 


65,114 


RNA 


Michigan 


147,452 


110,706 


132,954 


168, 125^ 


DNA 


« RNA 


Minnesota 


55,500 


RKA 


55,138 


DNA 


DNA 


RNA 


Mississippi 


266,190 


- 318,424 


198, 385^ 


202, 235^ 


183, 917^ 


147,886 


Missouri 


128,878 


119,767 


102, 724^ 


86,367 


80,943 


72,155 


Montana 


12,081 


7,420 


8,441 


10,213^ 


7,905 


RNA 


Nebraska 


38,758 


78,229 


68,498 


52,921 


38,524 


41 , 758 


Nevada 


1,958 


1,329 


1,443^ 


2,839^ 


1,903 ^ . 


RNA . 


New Hampshire 


DNA 


' DNA 


6,508^ 


3,006^ 


DNA 


RNA 


New Jersey 


133,149 


78,303 


86,524 


93,926 


88,694 


RNA * 


New V Mexico 


50,228 


51,064 


54,041^ 


30,802^ ' 


22,860^ 


RNA 


New York 


DNA 


DNA 


944,766 


435,955 


RNA 


RNA 


N- Carolina 


273,386 


260,582 


263,908 


225,595 


173,043 


RNA 


N« Dakota 


63,875 


49,036 


38,784 


27,432. 


23,998 


23,998 


Ohio 


165,047 


159,239 


140, 261^ 


132, 928^''* 


123,340 


122,629 


Oklahoma 


166,852 


RNA 


156,712^ 


9D,646 


DNA 


DNA 


Oregon 


32,494 


: 34,445 


42,254 


37,52l2 


44,007 


RNA 


Pennsylvania 


298,178 


265,556 


DNA 


339, 567^ 


334, 580^ 


• DNA 


Rhode; Island 


16,705 


•16,84^3 


15,993 


12,805 


17,604^ 


- RNA 


S. Carolina 


320,128 


. ;3l3",!i:)ii \ 


236, 157 


165,393 


152,613 


121,370 


S. Dakota 


36,865 




- ."37,6d4 


30,882 


22,467 ' 


22,551 


Tennessee 


222,877 


, DN^\ . ■ 


223,7781 


169, 651^ 


161,736 


RNA 


Texas 


RNA 


4^.7, • 


227, 248^ 


429, 257^ 


392, 317^ 


368,646 


Utah 


12,902 


iO*y79\ 

14:^^39 


14,786^ 


12,284^ 


RNA 


, RNA 


Vermont 


14,471 


13,937 


13,908 


14,293^ 


9,500^ 


Virginia 


148,310 


142,41^^ 


130,667 


129,070 


136,257 


118,100 


Washington 


62,491 


72,054' 




56,028^ ...^ 


DNA 


RNA 


W. Virginia 


95,493 


88,871 


83,572 


^' 68,620 


49>4^-^ 


RNA 


Wisconsin 


65,667 


63,101 


57,855 


57,174 


54,799 ' 


RNA 


Wyoming- 


20,912 


16,289' 


10,298 


9,289 


8,289 


RNA 


American Samoa 


DNA 


DNA 


RNA 


DNA 


1,360: ^ 


1,700 


BIA 


DNA 


DNA 


DNA 


DNA 


22,180 


25,328 


Guam 


7,388 


5,517 


RNA 


RNA 


1,430 


" DNA 


Puerto Rico 


RNA 


679,437 


220,585^ 


RNA 


RNA 


RNA 


Virgin Islands 


RNA 


RNA 


RNA 


RNA 


RNA 


DNA 


Range 


1,958 - 


1,329 


1,443 - 


2,839 - 


1,360 - 


1,700 - 




664,271 


777,634 


944,766 


435,955 


392,317 


596,921 


Number Reporting 46 


43 


46 


46 


42 


24 



DNA« Data Not Available R^A== Report Not Available v 

1. This figure may be a duplicated or unduplicated count . 2. An estimated figure. 
3. An approximate figure. -4. This figure may include nonpublic participants. 
5. Thia figure may include summer participants. 6. This figure represents the 



total enrollment in Title I schools. It may also represent the number of Title I 
participants. 
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FY 73, American Samoa had the smallest ntimber of participants and Texas had 
the largest. In FY 74, American Samoa again had the smallest number of partic- 
ipants and California had the largest. 

Far more states had a decreasing number of total participants than ha4 a 
relatively stable or increasing number over the last four fiscal years. ^ Al- 
though the explanation for this trend is unclear, it may be due to an attempt 
by several states to concentrate more funds on smaller numbers of children. 
This explanation is supported by an increase in mean per-pupil expenditures 
over the last four fiscal years (Table 31.) 

The percentages of nonpublic participants for each state reporting such 
data are presented in Table 17. The data for FY 69 and FY 70 have been re- 
produced from the Wargo report. Both duplicated and unduplicated counts were 
^ used for this breakdown, since it was assumed that the amount of duplication 

would not vary significantly between public and nonpublic participants. Of 
the 40 states that reported the number of nonpublic participants for 1971, 
Oklahoma had the smallest percentage, 0.2, and New York had the largest, with 
16.6 pe.rcent of their Title I participants from nonpublic schools. For FY 72, 
Alaska had the smallest proportion of nonpublic students, while Massachusetts 
had the largest. Tennessee had the smallest percentage of nonpublic students, 
and Massachusetts again had the largest in FY 73. In FY 74, Georgia reported 
the smallest proportion of nonpublic students and Nebraska reported the largest. 

Over the six fiscal years, there was a decrease in the number of State 
Reports providing information on public versus nonpublic participation and the 
explanation for this is not clear. 

There is considerable variation in percentages of nonpublic participation 
both within and between states. Twenty-three states showed a decrease in the 
proportion of nonpublic participation over the six-year period, several states 
had no identifiable pattern or stayed about the same, and only four states 
showed an increase. Data on the percent of eligible nonpublic students served 
by Title I are not available in the State Reports. 

Table 18 provides data on the percentages of Title I participants 
enrolled in regular, summer, and yearlong sessions in FY 71, 72, 73, and 74. 
This breakdown uses both duplicated and unduplicated figures. Only about one- 

6X 
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TABLE 17 



Percentage of Nonpublic School Children 
Served by Title I Presented by Fiscal Year 

State or 



Territory 


FY 69 


FY 70 


FY 71 


FY 72 


FY 73 


FY 74 


Alabama 


1.0 


0.7 


DNA 


DNA 


DNA 


DNA 


Alaska 


8.4 


3.1 


2.ll 


0,0^ 


DNA 


RNA 


Arizona 


DNA . 


6.6 


4.1 


3.8 


4.6 


3.4 


Arkansas 


0.6 


0.8 


0.8 


1.0 


0.4 


DNA 


California 


6.0 


3.6 


3.8 


3.3 


2.7 


2.3 


Colorado 


6.0 


6.2 


4.61 


3.6 


2.2 


3.7 


Connecticut 


10.3 


.10.7 


7.0 . 


5.0 


5.1 


5.4 


Delaware 


5.5 


RNA 


4.4 


1.4 


5.0 


4.8 


Wash. D. C. 


RNA 


DNA 


DNA 


5.0 


4.5 


DNA 


Florida 


5.0 


2.9 


3.5 


3.0 


3.4 


DNA 


Georgia 


0.6 


RNA 


0.5 


0.3 


0.3 


0.5 


Hawaii 


6.0 


3.5 


1.3 


0.8 


1.0 


RNA 


Idaho 


1.9 


2.0 


2.3 


0.8 


1.8 


0.8 


Illinois 


7.9 


RNA 


8.1 


10.3 


3.8 


4.8 


Indiana 


6.3 


3.9 


6.2 


4.9 


DNA 


DNA 


Iowa 


10.8 


9.9 


8.7 


9.9 


9.2 


RNA 


Kansas 


10.2 


4.4 


3.7 


3.7 


3.6 


RNA 


Kentucky 


DNA 


DNA 


DNA 


RNA 


1.1 


1.9 


Louisiana 


4.5 


6.0 


5.1 


DNA 


3.1 


RNA 


haine 


5.7 


6.5 


DNA 


5.2 


RNA 


4.7 


Maryland 


RNA 


5.1 


1.9 


2.1 


DNA 


RNA 


Massachusetts 


16.1 


17.5 


12.0 


13.0 


13.1 


RNA 


Michigan 


10.0 


7.7 


6.0 


DNA 


DNA 


RNA 


Minnesota 


9.0 


RNA 


8.9 


DNA 


DNA 


RNA 


Mississippi 


0.5 


0.2 


DNA 


DNA 


DNA 


DNA 


Missouri 


5.3 


4.2 


3.7 


4.7 


4.4 


5.6 


Montana 


12.3 


5.7 


6.2 


6.4 


3.9 


RNA 


Nebraska 


5.8 


4.0 


5.3 


10.5 


DNA 


9.6 


Nevada 


" 3.4 


4.4 


DNA 


DNA 




Ri'JA 
RNA 


New Hampshire 


DNA 


DNA 


DNA 


DNA 


DNA 


New Jersey 


7.8 


10.4 


12.0 • 


10.6 


11. 2^ 


RNA 


New Mexico 


8.2 


8.8 


5.4I 


5.4 


4.7 


RNA 


New York 


DNA 


DNA 


16.6 


DNA 


RI^IA 


RNA 


N. Carolina 


0.4 


0.5 


0.4 


0.6 


0.5 


.RNA 


N. Dakota 


8.9 


9.4 


6.9 


4.3 


4.5 


4.5 


Ohio 


4.5 


4.5 


4.8 


4.3 


4.3 


DNA 


Oklahoma 


0.6 


RNA 


0.2 


DNA 


DNA 


DNA 


Oregon 


5.5 


3.7 


3.8 


DNA 


4.2 


RNA 


Pennsylvania 


15,8 


11.8 


DNA 


9.9 


11.9^ 


DNA 


Rhode Island 


12.5 


10.2 


9.2 


12.4 


9.5 


RNA 


S. Carolina 


0.5 


0.4 


0.8 


0.8 


0.7 


0.6 


S. Dakota 


9.6 


9.0 


8.7 


8.9 


3.6 


4.0 


Tennessee 


2.0 


DNA 


0.8^ 


0.6l 


0.l3 


RNA 


.Texas 


RNA 


2.4 


2.32 


DNA 


1.5 


1.9 


Utah 


0.6 


0.5 


DNA . 


DNA 


DNA 


RNA 


Vermont 


0.7 


6.4 


6.8 


5.7 


6.8 


DNA 


Virginia 


DNA 


DNA 


DNA 


DNA 


DNA 


DNA 


Washington 


5.4 


6.7 


6.5 


1.7 


DNA' 


RNA 


W. Virginia 


1.7 


1.5 


DNA 


DNA 


DNA 


RNA 


Wisconsin 


8.0 


5.6 


8.2 • 


7.1 


6.4 


RNA 


Wyoming 


4.2 


3.2 


1.9 


i.a 


2.8 


RNA 


American Samoa 


DNA 


DNA 


RNA 


DNA 


DNA 


DNA 


BIA 


DNA 


DNA 


DNA 


DNA 


DNA 


DNA ' 


Guam 


14.6 


9.1 


RNA 


RNA 


4.2 


DNA 


Puerto Rico 


RNA 


1.1 


DNA 


RNA 


RNA 


RNA 


Virgin Islands 


RNA 


RNA 


RNA 


RNA 


RNA 


DNA 


Range : 


0.4 - 
16.1 


0.2 ~ 
17.5 


0.2- 
16.6 


0.0 - 
13.0 


0.1 - 
13.1 


0.5 - 
9.6 


Number Reporting 44 


42 


40 


.36 




16 



Data Not Available RNA= Report Not Available ^Percent of Title I particip, 



on an approximate figure. 2. This figure may be based .on actual data 

a projection from a sample. 3. This figure may include summer participation. 
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third of the State Reports contained data on participation by school session 
However, among those states providing such data, there was again considerable 
variation. As one would expect, all the states reporting such data have a 
higher proportion of regular year participants than either summer or yearlong. 
This emphasis may appear to be exaggerated, however, by figures from states 
that reported unduplicated counts and did not inlcude a breakdown for year- 
long participants. In these cases, presumably students enrolled for both 
regular and summer terms would have been reported only in the regular term 
category*' This difference in categorization should be taken into considera- 
tion in looking at the wide ranges in school session participation rates be- 
tween states. 

Although variations within states tend to be rather small, most 
of the changes were in the direction of larger regular year percentages 
and correspondingly smaller percentages for summer and yearlong sessions. 

Table 19 presents breakdowns of Title I participants by grade level 
by year, and Tables 20, 21, 22, and 23, present grade level data by state 
for FY 71 through FY 74. These percentages include both public and non- 
public participants, and are based on data labeled or presumed to be un- 
duplicated. There is little variation from year to year in the suimnary 
data in Table 19. The highest proportions of participants are in the early 
elementary grades, with grades 1, 2, and 3 accounting for approximately 
half of the participation for each of the three years. This is true for 
most of the states included in Tables 20 through 23, although there is 
considerable between-state variation, lliis picture is consistent with data 
in the Wargo report showing participants by grade band. However, since the 
number of states reporting participation by grade levels and grade bands 
is small, it is difficult to draw a national picture of which grades are 
receiving the most attention. 

Program participation and expenditures by state by fiscal year are 
shown in Tables 24, 25, 26, and 27. Percentages of participants were de- 
rived from duplicated counts and therefore total to more than 100. 

There is considerable variation among states; however, reading and 
language arts consistently serve large percentages of participants. 
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TABLE 19 



Percentage of Participants in Each Grade Level 
Presented by Year 





FY 71 


FY 72 


FY 73 


FY 74 


Prekindergarten 


1.8 


2.5 


1.8 


1 1 


Kindergarten 


8.5 


8.3 


7.6 


7 ft 
/ • o 


1 


12.3 


13.1 


13. 5 




2 


13.4 


14.1 


14.5 


X J • J 


3 


11.5 


13.1 


13. 6 


1 A 

X J • H 


4 


10.6 


11. 0 


11 9 


X ^ • u 


5 


9.8 


9.8 


1 n 


11 n 


6 


9.0 


8.3 


9.3 


9.6 


7 


5.9 


5.6 


5.1 


5.7 


8 


5.1 


4.4 


4.0 


4.1 


9 


4.4 


2.8 


2.9 


3.0 


10 


3.0 


1.8 


1.6 


1.9 


11 


2.2 


1.4 


1.1 


1.2 


12 


1.9 


1.1 


0.7 


0.7 


Ungraded 

Out of School 


0.4 
0.0 


2.1 

0.4 


1.6 
0.1 


1.1 
1.2 


Number of States 
Reporting 


13 


14 


18 


16 




TABLE 20 



State 



Percentage of Participants in Each Grade Level 

Fiscal Year 1971 



Pre-K 



Grade 



K 



Arizona 0.1 5.0 11.2 10.8 10.7 8.9 8.4 7.5 

California 0.7 11.3 13.6 13.7 13.4 12.3 11.4 10.1 

Florida 6.9^ 12.3 14.6 13.8 8.9 7.4 9.4 

Kansas 7.4 9.0 11.2 11.4 10.7 9.9 8.5 

Maryland 1.0 10.0 7.8 22.9 20.8 13.4 10.3 8.0 

Massachusetts 5.2 9.7 16.9 16.1 14.3 11.2 9.5 8.2 

Montana 0.3 1.5 7.7 10.6 11.2 10.5 10.2 8.5 

New York 2.7 9.5 13.2 13.5 10.2 10.3 9.7 8.9 

Oklahoma 8.2 8.4 9.4 9.6 9.5 9.5 

Oregon 0.9 5.0 10.4 11.6 11.9 9.6 7.8 6.8 

Rhode Island 1.6 5.5 16.4 17.8 14.5 11.5 8.6 7.3 

South Dakota 0.5 10.9 8.1 9.6 10.8 10.2 9.5 11.3 

Wyoming 0.2 4.7 10.6 10.7 11.3 12.1 11.6 10.7 



Range 



0.1 - 1.5 - 7,7 - 8.4 
5.2 11.3 16.9 22.9 



9.4 - 8.9 - 7.4 - 6.8 
20.8 13.4 11.6 11.3 



Includes Pre-K 



7G 



1 




TABLE 20 




- 














icipants in Each Grade Level, 
; Fiscal Year 1971 


Presented 


by 


State for 










Grade 


1 3 


4 


5 


6 


7 


8 


9 


10 


11 


12 


Un- 

Graded Other 


).7 


8.9 


8.4 


7.5 


7.3 


6.9 


10.6 


5.5 


4.0 


2.7 


0.4 


3-4 


12.3 


11.4 


10.1 


3.0 


2.2 


3.9 


2.1 


1.0 


0.6 


0.6 


3.8 


8.9 


7.4 


9.4 


8.7 


5.9 


3.1 


2.8 


1.4 


1.1 


2.6 




10.7 


9.9 


8.5 


9.5 


7.0 


4.6 


3.5 


2.9 


2.5 


2.0 


5.8 


13.4 


10.3 


8.0 


1.6 


3.9 


0.0 


0.1 


0.0 


0.0 




if o 

^- 3 


11. 2 


9 . 5 


8.2 


3 . 3 


2 . 4 


1.1 


1.0 


r\ "J 

0. 7 


0.5 




L.2 


10.5 


10.2 


8.5 


9.4 


7.9 


8.2 


6.5 


3.4 


2.8 


0.8 0.4 


f).2 


10.3 


9.7 


8.9 


5.9 


5.2 


4.1 


2.8 


2.0 


1.7 


0.2 


M 


9.6 


9.5 


9.5 


10.4 


9.8 


8.5 


6.1 


5.5 


5.0 




i..9 


9.6 


7.8 


6.8 


7.2 


6.0 


6.3 


5.5 


6.2 


4.9 




4- 5 


11.5 


8.6 


7.3 


5.9 


2.0 


2.6 


1.8 


0.7 


0.2 


3.6 


i).8 


10.2 


9.5 


11.3 


7.5 


6.6 


4.0 


3.2 


2.9 


4.1 


1.0 


1.3 


12.1 


11.6 


10.7 


7.2 


6.2 


4.9 


2.0 


1.5 


1.3 


5.0 


J. 4 - 
j).8 


8.9 
13.4 


7.4 - 
11.6 


6.8 - 
11.3 


1.6 - 
10.4 


2.0 
9.8 


0.0 - 
10.6 


0.1 - 
6.5 


0.0 - 
6.2 


0.0 - 
5.0 


0.2 - 
5.0 
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TABLE 21 

Percentage of Participants in Each Grade Level, Presented 

Fiscal Year 1972 



State 



\ 



Pre-K 



K 



Grade 



CoXorado 5.6^ 10.3 11.5 11. 1 9.4 8.1 7.9 6.9 

Hawaii 3.0 5.2 8.3 8.9 10.0 10.0 9.6 9.4 8.5 

Kansas 1.8 4.2 9'. 8 12.5 11.9 11.9 11.2 9.8 7.9 

Maine 8.5 14.4 16.3 15.8 14.5 12.8 10.9 2.1 

Maryland 3.2 13.0 20.5 19.1 15.2 7.9 5.8 2.9 4.4 

Massachusetts ,4.8 10.2 17.1 17.0 14.8 11.0 9.5 8.4 2.2 

Missouri 0.8 2.7 8.3 11.9 12.6 13.2 12.8 11.2 8.4 

Montana^ 0.8 1.3 8.1 8.9 10.7 11.6 10.6 8.5 9.2 

North Dakota 12. 2^ 8.9 10.4 10.6 11.5 11.4 10.7 7.1 

Rhode Island 0.2 5.0 14.9 16.2 15.1 9.4 8.2 7.8 7.3 

South Dakota 0.8 12.9 9.5 11.2 11.7 11.7 10.4 10.0 7.0 

Vermont 2.1 6.0 13.3 13.5 13.5 10.7 9.4 8.7 7.3 

Wisconsin" 7.8 13.0 16.2 15.3 13.2 9.6 7.2 5.2 2.4 

Wyoming 5.7 11.0 10.4 12.1 12.5 12.2 10.7 6.3 



Range 



0.2 
7.8 



1.3 
13.0 



8.1 
20.5 



8.9 
19.1 



10.0 
15.8 



7.9 

14.5 



5.8 
12.8 



2.9 
11.2 



2.1 
9.2 



1. Includes Pre-K' 

2. It is not clear whether this is a duplicated or unduplicated count. 
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TABLE 21 

irticipants in Each Grade Level, Presented by State for 
Fiscal Year 1972 



Grade 



10 



11 



12 



Un- 
Graded 



Other 



11.1 

10.0 
11.9 
15.8 
15.2 
14.8 
12.6 
10.7 
I 10.6 
15.1 
11.7 
13.5 
13.2 
12.1 



9.4 
10.0 
11.9 
14.5 

7.9 
11.0 
13.2 
11.6 
11.5 

9.4 
11.7 
10.7 

9.6 
12.5 



8.1 
9.6 
11.2 
12.8 
5.8 
9.5 
12.8 
10.6 
11.4 
8.2 
10.4 
9.4 
7.2 
12.2 



7.9 
9.4 
9.8 

10.9 
2.9 
8.4 

11.2 
8.5 

10.7 
7.8 

10.0 
8.7 
5.2 

10.7 



6.9 
8.5 
7.9 
2.1 
4.4 
2.2 
8.4 
9.2 
7.1 
7.3 
7.0 
7.3 
2.4 
6.3 



7.1 
6.7 
5.8 
1.5 
3.2 
1.5 
6.6 
7.9 
5.3 
4.0 
5.3 
6.4 
2.2 
7.2 



5.4 
5.8 
4.4 

0-.2 
0.7 
4.2 
7.6 
3.4 
4.6 
3.6 
3.5 
1.7 
5.0 



3.8 
5.3 
2.8 

0.1 
0.6 
1.8 
5.9 
3.0 
2.3 
2.3 
1.6 
1.7 
4.0 



3.4 
4.7 
2.5 

0.0 
0.5 
1.3 
3.9 
2.7 
1.1 
2.1 
1.2 
1.0 
1.2 



2.6 
4.4 
2.1 

0.0 
0.4 
0.9 
2.3 
2.4 
0.9 
1.3 
1.1 
0.7 
1.1 



6.9 

1.4 
3.2 
4.2 
0.4 
3.2 
1.1 

3.0 
0.3 
1.5 

0.5 



0.5 



1.3 



2.7 



10.0 
15.8 



7.9 
14.5 



5.8 
12.8 



2.9 
11.2 



2.1 
9.2 



1.5 
7.9 



0.2 - 
7.6 



0.1 
5.9 



0.0 
3.0 



0.0 
4.4 



0.3 
6.9 



0.5 
2.7 



t unduplicated count. 
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TABLE 22 














Percentage of Participants in 


Each Grade 


Level, 


Presente 
















Fiscal Year 1973' 








- 












Grade 




' 




State 


Pre-K 


K 


1 


2 


3 


4 


5 


6 


7 




Arizona 


2.0 


6.0 


13.5 


12.6 


11.5 


9.0 


7.8 


7.5 


5.4 . 




California 


1.9 


11.8 


13.9 


14.2 


14.2 


12.7 


11.9 


10. 7 


2.0 




Colorado 




6.6 


1 11.8 


12.8 


11.1 


8.8 


8.2 


7.5 


7.3 ; 




Idaho 




8.6 


10.4 


11.1 


11.6 


10.9 


10.3 


10.0 


6.4 




Kansas 




6.7 


1 11.0 


15.1 


12.5 


11.2 


10.5 


9.0 


8.6 




Massachusetts 


4.9 


10.2 


18.1 


17.6 


14.7 


11.2 


8.7 


6.9 


1.7 




Mississippi^ 


0.8 




17. 0 


16.3 


14.1 


12.8 


12.0 


10.8 


6.4 




Missouri 


1.0 


2.7 


9.0 


12. 7 


13.6 


13.4 


12.9 


11.0 


8.1 ■ 




Montana^ 


0.3 


1.5 


7.7 


10.6 


11.2 


10.5 


10.1 


8.5 


9.4 




New Jersey^ 


3.7 


10.6 


19.8 


21.3 


19.9 


10.4 


6.1 . 


4.4 


1.6 




North Dakota 


12.4 




9.0 


11.0 


11.6 


11.1 


10.9 


10.8 


7.2 




Rhode Island^ 


0.4 


3.9 


11.4 


15.6 


16.9 


15.1 


8.3 


6.6 


7.0 




South Dakota 


0.7 


13.4 


8.4 


11.0 


12.2 


13.2 


12.1 


10.8 


7.5 




Texas ^ 


1.1 


7.1 


11.2 


13.2 


12.1 


12.1 


10.1 


9.7 


7.3 




Wisconsin 


9.2 


14.7 


17.1 


16.1 


14.4 


8.6 


5.6 


3.8 


2.0 




Wyoming 




5.9 


10.8 


12.8 


12. 3 


12.1 


12.2 


10.5 


6.5 




American Samoa^ 






















Guam 




32.1 


30. 7 


18.9 


9.5 


7.8 


X).8 


0.2 






Range 


0.3 - 


1.5 


- 7.7 


- 10.6 - 


9.5 


- 7.8 


- 0.8 - 


n 9 


1.6 - ; 






12.4 


32.1 


30.7 


21.3 


19.9 


15.1 


12.9 


11.0 


9.4 




1. Includes Pre-K. 






















2. It is not clear 


whether 


this 


is a duplicated or 


unduplicated 


count. 






i 


3. It is not clear 


whether 


these 


figures 


represent 


projections from sample 


data or 


actual E 


1 

{ 


It is also not 


clear whether 


this is 


a duplicated or 


unduplicated count. 






i 

! 






8( 


1 















TABLE 22 



jof Participants in Each Grade Level, Presented by State for 
Fiscal Year 1973 









Grade 










































3 


4 


5 


6 


7 


8 


9 


10 


11 


12 


Un- 

Graded Other 


i 


11.5 


9.0 


7.8 


7.5 


5.4 


4.5 


11.3 


4.7 


2.4 


1.8 


0.0 


I 

i 


14- 2 


12. 7 


11.9 


10.7 


2.0 


1.2 


3.3 


1.7 


0.7 


0.3 


0.2 


\ o 
1 ° 


11. 1 


8.8 


8.2 


7.5 


7.3 


6.8 


5.2 


3.0 


2.9 


2.2 


5.8 


1 


11.6 


10.9 


10 . 3 


10.0 


6.4 


5.2 


4.6 


4.0 


4.1 


2.7 




1 


12. 5 


■ 11.2 


10.5 


9.0 


■ 8.6 


5.5 


4.2 


1.9 


1.4 


1.1 


1.1 


• 0 


14. 7 


11. 2 


8. 7 


6.9 


1.7 


1.3 


1.0 


0.7 


0.5 


0.5 


0.5 1.6 


.3 


14.1 


12.8 


12.0 


10.8 


6.4 


4.9 


2.1 


1.1 


0.9 


0.7 




l7 


13.6 


13.4 


12.9 


11.0 


8.1 


6.0 


3.0 


1.4 


1.0 


0.6 


3.5 


16 


11.2 


10.5 


10.1 


8.5 


9.4 


7.9 


8.2 


6.4 


3.4 


2.8 


0.8 0.4 


\z 


19.9 


10.4 


6.1 


. 4.4 


1.6 


1.2 


0.3 


0.2 


0.3 


0.0 






11.6 


11.1 


10.9 


10.8 


7.2 


6.4 


3.0 


2.9 


2'.1 


1.6 




|6 


16.9 


15.1 


8. 3 


6.6 


7.0 


5.3 


3.7 


2.4 


0.8 


0.3 


2.3 


fO 


12.2 


13.2 


12.1 


10.8 


7.5 


5.8 


1.8 


1.0 


0.5 


0.4 


1.2 


i2 


12.1 


12.1 


10.1 


9.7 


7.3 


6.4 


2.5 


1.5 


1.2 


1.0 


3.3 


:i 


14.4 


8.6 


5.6 


3.8 


2.0 


1.7 


1.7 


1.4 


1.0 


0.8 


1.9 


;8 


12.3 


12.1 


12.2 


10.5 


6.5 


5.4 


4.6 
27.9 


0.9 
30.5 


0.8 
25.6 


0.6 
16.0 


4.4 ; 

0.1 \ 



9.5 



7.8 



0.8 



6 - 9.5 - 7.8 ~ 0.8 
3 19.9 15.1 12.9 



0.2 

0.2 
11.0 



1.6 - 1.2 - 0.3 - 0.2 - 0.3 - 0.0 ^ 0.0 - 0.1 
9.4 7.9 27.9 30.5 25.6 16.0 5.8 1.9 



»d or unduplicated count. 

jsent projections from sample data or actual participation per grade level 
.icated or unduplicated count- 
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TABLE 23- 



ERIC 



Percentage of Participants in Each Grade Level, Present; 

Fiscal Year 1971 



Grade 



State Pre-K K 



Arizona 1.5 5.1 11.7 12.9 12.3 10.1 9.5 7.7 

Arkansas 0.1 1.7 6.3 10.7 11.4 12.3 11.6 12.0 

California 0.7 14.1 15.5 13.7 13.2 10.9 .10.0 9.3 

Colorado 1.6 5.6 12.7 13.1 11.7 8.8 8.4 7.6 

Florida 0.3 5.9 15.8 18.4 15.5 11.2 10.5 8.7 

Georgia 7.4^ 10.9 13.8 13.3 11.8 11.0 9.9 

Idaho 0.0 5.1 15.0 11.6 14.5 11.6 11.3 14.2 

Kentucky 0.0 3.1 11.7 14.3 14.6 13.5 12.1 10.6 

Maine 0.5 9.3 15.7 16.3 16.1 14.4 13.6 10.6 

Mississippi 0.6 0.0 14.5 14.1 14.0 13.5 13.3 11.6 

Missouri 1.3 4.6 7.6 13.6 14.1 13.9 13.5 11.8 

North Dakota 11.9 0.0 9.7 11.9 13.2 12.6 11.1 10.6 

Ohio 5.7 4.0 14.5 18.5 ik.l 11.2 8.2 6.1 

South Carolina 0.3 4.4 7.5 9.8 11.3 11.9 11.8 10.3 

South Dakota 1.0 10.9 10.0 12.3 12.8 13.8 12.6 11.6 

Texas 0.8 8.0 9.8 11.6 12.7 13.0 11.7 9.1 

Range 0.0 - 0.0 - 6.3 - 9.8 - 11.3 - 8.8 - 8.2 - 6.1 - 0.0 

11.9 14.1 15.8 18.5 16.1 14.4 13.6 14.2 10.4 



Includes Pre-K 
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TABLE 23 



cipants in Each Grade Level, Presented by State for 
Fiscal Year 1971 



i Grade • 




4 


5 


6 


7 


8 


9 


10 


11 


12 


Un- 

Graded • 


Other 



2.3 


10.1 


9.5 


7.7 


5.4 


5.1 


11.5 


4.4 


1.8 


0.8 


0.0 


0.0 


1.4 


12.1' 


11.6 , 


12.0 


10.4 


7.6 


4.6 


3.4 


2.4 


1.8 


2.7 


0.9 


3.2 


10.) 


10.0 


9.3 


3.1 


2.0 


3.3 


2.0 


1.1 


0.6 


0.4 


0.0 


L.7 


8.8 


8.4 


7.6 


7.3 


6.3 


3.8 


2.6 


2.5 


1.6 


6.3 


0.0 


5-5 


11.2 


10.5 


8.7 


6.7 


2.5 


1.6 


1.5 


0.4 


0.1 


1.0 


0.0 


3-.3 


11.8 


11.0 


9.9 


9.3 


5.8 


; 3.4 


1.6 


1.1 


0.7 


0.0 


0.0 




' 11.6 


11.3 


14.2 


5.6 


4.5 


3.3 


1.4 


1.1 


0.7 


0.0 


0.0 


^.6 


13.5 


12.1 


10.6 


8.6 


6.8 


2.7 


1.0 


0,5 


0.5 


0.0 


0.0 


6.1 


14.4 


13.6 


10.6 


0.0 


0.0 


0.0 


0.0 


0.0 


0.0 


0.0 


3.5 


if.O 


13.5 


13.3 


11.6 


6.2 


4.6 


2.2 


1.2 


0.5 


0.3 


0.0 


3.3 


Ll 


.L3.9 


13.5 ■ 


11.8 


7.9 


6.1 


2.0 


1.4 


0.7 


0.4 


1.1 


0.0 


3.2 


12.6 


11.1 


10.6 


7.1 


5.2 


2.6 


1.7 


1,2 


1.1 


0.0 


0.0 


6.1 


11.2 


8.2 ■ 


' 6.1 


3.0 


1.6 


0.9 


1.1 


0.8 


0.3 


1-7 


6.0 


t.3 


11.9 


11.8' 


10.3 


1.1 


6.4 


, 6.1 


5.0 


3.7 


2.1 


0.6 


0.8 


Ls 


13. a 


12.6 


11.6 


6.5 


5.2 


1.2 


0.4 


0.2 


0.0 


0..0 


1.3 


2.7 


13.0 


11.7 


9.1 


6.8 


5.4 


2-5 


1.8 


1.2 


1.0 


2.8 


1.8 



8.8 - 8.2 - 6.1 - 0.0 - 0.0 
14.4 13.5 14.2 10.4 7.6 



0.0 - 0.0 - 0.0 - 0.0 - 0.0- 0.0- 
11.5 5.0 3.7 2.1 6.3 6.0 
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TABLI 

Psrcentage of Participa 
for Programs by State 



Alabama 

Instructional Par. hlxp . 


Arizona 
Par. Exp. 


Arkansas 
Par. Exp. 


Idaho 
Par ExD . 


Kansas 
Par. Exp. 


Louisiana 
Par, Exp. 


1 

P< 


Art 


0.9 


5.1 


0.8 


9.4 


1.8 


0. 8 


6.6 


4.0 


0.6 




Business Education 


0.2 


1.0 


0.5 


1.0 


0.7 


'0 . 2 


. 0. 1 


1.5 


0.3 




Cultural Enrichment 


4.5 


21.4 


2.1 


0.6 


0.7 


0. 2 


2.4 


3.5 


0.6 




English-Reading 


45.0 


70.2 


35.4 


21.1 


19.6 


24. 4 


57. 1 


71.9 ' 


34.5 


: 6i 


English-Speech 


0.9 


23.3 


4.6 


0.1 


0.1 






L.0 


0,6 




English-Other L.A. 


2.5 


20.0 


5.5 


4.1 


3.3 




H . -L 


6.1 


1.3 


i; 


E . S . L 


0.0 


20.1 


8.4 










0.4 


0.4 




Foreign Language 




0.6 


0.2 






n 9 




2.5 


0.1 




Home Economics 


0.2 ' 


0.5 


0.2 


0.3 


0.2 


n f\ 




1.6 


0.3 




Industrial Arts 


0.3 


0.9 


0.4 


1.6 ^ 


2.0 


1 f\ 

i. . o 




2.4. 


0.8 




Mathematics 


8.0 


11.3 


1.5 ' 


4.5 


2.9 


9 n 

4 . U 


^f\ 9 


28.9 


4.4 


1 < 


Music , 


2.1 


8.9 


1.2 


26.7 


5.1 






10.7 


1.6 


1 j 

•LI 


PE, Health, Rec. 


3.7 


13.5 


1.5 


11.3 


2.0 


0.3 


8.3 


17.6 


2.3 


1 


Natural Science 


1- 3 


2 . 9 


0. 2 


0 . 9 


U . o 


0.5 


1.0. 


i/ . O 


1 Q 




Social Science 


1.3 


4.2 


0.4 


1.0 


0.8 


1.1 


0.1 


21.1 


1.2 




Other Voc. Ed. 


0-7 


0.6 


0.3 


0.1 


0.1 


7.4 


0.4 


1.3 


1.3 




Special Activities 
for Handicapped 


3.9 


1.6 




2.1 


6.9 


2.5 


4. 1 


5.0 


6.6 




K & PreK 


4.3 


* 3.2 


5.0 


0.7 


1.3 


19 ^ 


9 7 


4.3 


2.0 




Other 


1.0 


6.0 


1.5 


10.8 


16.6 


n f\ 

u . o 


9 9 
z . z 


28.1 


1.8 




Support ive 






















Attendance 


0.7 


10.5 


0.9 


15.0 


0.8 


n n 
u . u 




10.9 


1.6 




Clothing 


0.0 


4.8 


0.1 






n 9 




0.1 


0.0 




Food 


1.9 


10.3 


0.7 


30.1 


i:l 




7 . 9 


4.3 


0.2 




Guidance Counseling 


5.5 


27.1 


10. 6 


37.7 


6.6 




22 - 1 


34.9 


5.9 




Health-Oental 


0.6 


3.3 


0.6 


9.5 


1.2 






5.6 


0.6 . 




Health-Medical 


1.5 


20.x 


1.4 


38.4 


3.9 




23. 5 


37.6 


3.1 




Library, Media Center 


4.6 






86,5 


9.5 


4.8 




9.3 


18.4 




Psycho. Services 


0.4 


21.4 


2.4 


68.0 


0.4 


2.8 


2.3 


4.2 


1.0 




\Social Work 


0.3 


7.0 


2.6 


32.3 


2.9 




7.7 


6.9 


1.2 




Speech Therapy 


0.4 


9.4 


1.7 


3.3 


1.6 


0.6 


3.4 


5.0 


2.4 




Transportation 


2.3 


2.5 


1.2 


1.3 


1.1 


1.6 




9.3 


1.1 




Spec. Services 
for Handicapped 


0.0 


19.8 


2.0 


0.0 


0.0 






2.1 


2.1 




Other 


0.9 


9.6 


1.6 


31.5 


4.6 


27.2 




1.9 


0.6 





Percentages total to more than 100 because a number of children parnicipated in more than one program, 
cated number of Title I participants was used to derive each program percentage. 
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pants and £;q)enditures 
k for Fiscal Year 1971 



f N. Dakota 



jPar. 


Exp. 


Par. 




1.9 


1.6 






0.4 






1.9 


^3:2 


52.7 


6.7 


jll .5 


5.8 


2.6 


[ 0.0 


0.0 


0.6 


!l9.4 


10.8 


3.3 


il8.8 


4.9 


3.0 


jl3.4 


3.4 


2.7 


! 1-7 


0.6 


1.9 


; 5.0 


2.2 


6.8 






0.4 


I 0.7 


3.1 


1.6 


;io.2 


8.2 





9.5 1.5 



14.1 4.0 



1.2 1.1 



Oklahoma 

Exp. 



19.4 



2.0 
0.0 
8.0 

19.5 
1.1 

20.5 
4.4 
0.3 
2.5 



Oregon 
Par. Exp. 



Rhode 
Island 
Par. J?xp_. 



S. Carolina 
. Pnr. Exp. 



S. Dakota 



Par. 



Virginia 
Pnr. r.xp. 



The undupli- 



7.9 


0.7 


1.3 


21.1 


1.9 


4.9 


1.6 


17.7 


1.0 


0.5 


0.0 


0.5 


2.1 


0.6 


0.5 


0.5 






32.5 


5.3 


2.7 


S.7 


0.9 


6.4 


3.0 


31.8 


4.7 


67.6 


51.7 


41.8 


75.3 


30.9 


46.3 


44.2 


98.4 


38.1 


4.6 


1.1 


1.1 


3.2 


0.5 


0.5 


1.6 






23.8 


7.0 


0.7 


7.0 


2.3 


4.3 


3.9 


35.72 


8.52 


0.4 


0.3 


6.6 


0.2 


0.1 


0.5 


0.5 






0.5 


0.0 




0.2 


0.1 


0.8 


0.2 


0.3 




2.4 


0.4 




0.3 


0.1 


0.0 


0.0 


1.0 


0.5 


2.6 


0.5 


1 9 
J. . ^ 


i. . 1 


U . U 


0 .-2 


""'0.2 


0.7 


0.4 


24.9 


6.7 


5.0 


28.1 


4.1 


15.4 


8.2 


32.2 


7.0 


4.0 


0.4 


1.0 


26.3 


2.5 


.1.6 


0.2 


20.5 


1.5 


15.2 . 


1.9 


1.6 


26.2 


1.6 


3.4 


0.5 


16.4 


2.2 


5.2 


0.7 


1.1 


39.9 


2.5 


1.2 


0.7 


12.4 


1 . 6 


6.1 


1.2 


1.5 


8.9' 


1.4 


2.3 


0.7 


6.9 


0.8 


2.8 


1.7 


0.0 


2.3 


2.5 


0.0 


0.0 


2.3 


0.9 


1.7 


1.2 


4.0 


3.8 


3.8 


0.9 


1.1 


2.9 


5.6 


4.4 


3.9 


1.8 


2.1 


4.0 


23.6 


8.7 


11.5 


8.7 


5.0 


1.4 


8.9 


38.1 


10.8 










4.2 


0.1 


2.6 


21.3 


1.1 


2.0 


0.3 


28.4 


2.5 


d.o 


0.0 


1.5 


3.3 


0.4 


0.5 


0.0 


6.8 


0.3 


12. 2 


0.5 


1.8 


39.6 


6.8 


2.5 


0.0 


55.5 


5.2 


12. 8 


4.4 


3.6 


24.6 


2.2 


9.1 


2.5 


20.1 


1.4 


2.9 


0.3 


0.2 


12. 2 


1 1 

X . X 


0 . 1 


n n 

u . u 






4.4 


0.7 


1.7 


55.0 


3.5 


8.8 


1.0 


40. 9^ 


2.2 


L0.6 


1.2 


0.3 


68.9 


5.7 


9.2 


0.8 




3.5 


2.2 


0.6 


0.4 


6.8 


0.6 


1.1 


0.2 


6.2 


0.8 


1.9 


2.2 


2.8 


13.8 


1.5 


1.8 


0.5 


10.0 


0.6 


1.0 


0.6 


0.9 


1.8 


0.5 


1-9 


1.2 


5.4 


0.8 


17. 1 


3.0 


2.3 


13.9 


1.4 


7.9 


3. A 


50.3 


2.4 


0.1 


0.0 


0.4 


0.5 


0.3 


0.1 


0.1 






0.0 


0.0 


0.6 


31.0 


3.8 


1.2 


0.8 


29.4 





Wyoming 
Pn r . Exp . 

16.7 



33.3 
49.9 



0.8 
0.6 



25.3 
16.8 
23.3 
19.5 
16.7 
2.6 

1.4 
48.5 



5.3 

15.5 
15.2 

40.8 

13.5 
1.1 

6.8 

Includes speech and ESL 
Includes dental 
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/ 



Instructiona l 



Alabama Arizona 
Par . Fxp . Par . Exp . 



/\rkansas 
Par. Exp. 



Colorado 
Par. Exp.- 



Idaho 
Par. Exp. 



Louisiana 
Par. Exp. 



TAB! 

Percentage of Par tic ij 
for Programs by State 



Maine 
Par. Exp. 



Masfi. Mi 
Par. Exp. Pa 



•Art 

Business Sducation 
Cultural Enrichment 
English Reading 
English-Speech 
English-Other L.A. 
ESL 

Foreign Language 
Home Economics 
Industrial Arts 
Mathematics 
Music 

PE, riealth, Rec. 
Natural Science 
Social Science 
Other Voc. Ed. 
Special Activities 



Supportive 







1 


1 


0. 


1 


0.0 






1 


0 


0 


6 


2 


3 






28. 9^ 


0 


0 


0 


-1 


0 


4 


0.2 






1 


4 


0 


1 












0 


7 


13 


6 


0 


2 


0.1 


10 


3 


4 


0 


0 


6 


2 


3 






43.8 


50 


5 


75 


3 


32 


5 


25.5 


50 


4 


46 


8 


20 


5 


67 


8 


34 


7 90. 3 


73.6^ 


0 


0 


17 


1 


0 


1 


0.1 














0 


4 


0 


3 




1 


8 


20 
17 


1 
2 


8 
0 


4 
9 


5.3 


21 


0 


39 


3 


1 


5 


5 


5 


2 


0 


12.6 
8.2 



0.0 





1 


0 


0.4 


0 


2 




0 


3 


0.2 














0.0 


1 


0 


1.5 


l' 


5 




1 


4 


1.8 




5 


0 


2 






16. 7 


9 


2 


9.9 


5 


1 




7 


8 


1.2 


41 


4 


14 


3 


15.7 


35.5 




1 


8 


0.3 


0 


0 




16 


0 


2.4 


2 


3 










0.1 


3 


6 


1.1 


0 


1 




58 


6 


5.8 


0 


5 


0 


1 


5.0 


20.7 


1.2 


2 


2 


1.9 


0 


9 


22. 9^ 


2 


7 


0.2 


2 


8 






4.1 . 


13. 7 


1.4 


2 


1 


2.1 


0 


9 


9.6 


4 


3 


0.9 


3 


0 


0 


6 




9.0 


0.0 


0 


4 


0.3 


0 


4 


5.6 


2 


4 


1.5 


0 


8 


1 


3 







for Handicapped 


3.9 


0 


7 


3 


0 


10 


3 7.8 


2 


5 


7.1 


5 


5 


9 


6 


8.1 


K and PreK 


6.8 


2 


6 


1 


5 


I 


9 


5 


9 


5.8 


2 


1 


1 


3 




Other 


4.4 


2 


7 


16 


2 


27 


6 17.3 


36 


6 


8.1 


26 


1 


10 


4 


4.0 



Attendance 


0 


7 


14 


9 


3 


1 


0 


1 






8 


1 


o.a 


16 


8 


1 


1 






Clothing 


0 


0 


0 


2 










0 


6 


0 


2 


0.0 


1 


4 


0 


0 


1 


0 


Food 


0 


3 


4 


7 


5 


3 


0 


3 


5 


0 


4 


1 


0.2 


2 


1 


0 


2 


5 


V 


Guidance Counseling 


2 


4 


18 


1 


44 


7 


5 


1 


43 


7 








26 


5 


5 


5 


11 


0 


Health-Dental 


0 


8 


6 


1 


9 


7 


1 


2 












5 


0 


0 


7 






Health-Medical 


1 


8 


22 


0 


38 


1 


3 


4 


12 


73 








42 


7 


2 


7 


9 


63 


Library, Media Center 


1 


4 






3 


4 


0 


2 


30 


4 


49 


6 


6.1 


66 


1 


7 


6 


2. 


9 


Psychological Services 


0 


3 


9 


7 


35 


4 


0 


5 


7 


5 


23 


1 


3.6 


5 


3 


1 


0 


3 


3 


Social Work 


0 


4 


11 


3 


38 


1 


3 


2 


12 


9^ 


0 


5 




10 


6 


1 


4 


6 


8 


Speech Therapy 


0 


2 


3 


1 


2. 


7 


1 


5 






2 


0 


0.2 


7 


4 


1 


7 


4 


9 


Transportation 


2 


0 


9 


6 


3 


8 


0 


7 


10 


2 


19 


1 


1.5 


9 


4 


0 


8 


11 


9 


Spec. Svs. for Handicapped 


0 


0 


0 


5 


0 


0 


0 


1 


2 


9 








1 


2 


1 


1 






Other 


2 


1 


2 


8 


32 


5 


3 


5 


8 


7 


239 


4 


29.4 


9 


6 


1 


4 


2 


0 



37.7^ 



13.1^ 



36.7 



5.1 



20.4 



1. Percentages total to more than 100 because a number of children participated in more than one program. The unduplicated number I 
ipants was used to derive each program percentage. ^ 



5. Includes Music, Dramatics, and Crafts 



6. Includes Food 



7. Includes all English 



8. Includes ESL and Other 
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Icipants and E q^enditures 
^atc cor Fiscal Year 1972 



Mississippi 
Par. 



N. Dakota 



Oklahoma 



Rhode Island S. Carolina 



S. Dakota 



Tennessee 



Virginia 



Wisconsin 



Wypming 



Exp. Par. 


Exp. 


Par. 


Exp. 


Par. 


Exp. 


Par. 


E^p. 


Par, 


Exp. Par. 


Exp. 


Par. 


Exp. 


Par. Exp. 


Par 


1.2 , 


2.6 


3.8 


7.2 






2.8 


0.3 


4.1 


1.4 


1 . 2 


14 . 2 


1.2 


5.7 


19. 7 


'^1 




0.6 


2.5 






0. 1 


0.0 


0.3 


0.1 


0.2 






0.1 




0.0 




3.2 


1.0 


4.1 


0. 1 


1.3 


0.7 


4.2 


2.1 


0.8 


32. 1 


3.2 


16. 3 


26.6 


47.4 


49.3 


49.6 


17. 9^^ 


68. 8 


54. 1 


87. 7 


34 . 9 


48.4 


40.2 


35.0 


99. 1 


41.9 


47.6 


79. 5 


0.7 










0.3 


2.8 


2.9 


2.5 


1.7 




2. 9 


6.2° 


3.4 




5 . 2 


10.9 


5.9 




6.6 


1.7 


17.4 


8.9 


8.2 


6.2 


15.9 


16.1 




8.9 












7.0 


9.6 






0.0 


0.0 




4.2 




1.1 


2.5 














0.0 


0.0 






0.0 


0.3 




0.7 




0.1 










0.1 


1.3 


0.0 


0.0 


0.0 


0.1 


2.8 


0.6 


0.1 




0.9 


0.6 


0.9 


4.2 


2.0 


1.7 


19.0 


0.2 






0.4 


2.6 


0.8 


0.2 




15.7 


12.4 


7.5 


5.6 


7.8 


4.2 


76.5 


7.6 


15.7 


7.6 


4.9 


52.4 


7.4 


15.5 


46.6 


2.3 


4.4 


7.6 


10.6 






1.9 


0.4 


0. 7 


0. 1 


2.0 


13.6 


1.0 


6.0 


20.9 


4.0 


1.9 


5.2 


6.7 




1.1 


1.7 


0.2 


1.8 


0.3 


1.5 


14.6 


1.9 


10.0 


30.2 


1.7 


1.0 


0.3 


1.1 




0.2 


9.2 


0.8 


0.6 


0.4 


0.9 


9.0 


1.1 


8.5 


25.0 


0.9 


2.1 


2.0 






0.7 


5.5 


0.7 


0.5 


0.2 


0.6 


4.4 


0.9 


4.3 


28.4 


0.9 




1.6 




0.7 




5.1 


2.6 


0.0 


0.0 


1.1 


1.1 


0.4 


0.3 


3.1 


J . o 


5. 3 


8.9 




3. 7 


7.8 


5 . 1 


4.1 


0.9 


0.6 


1.5 


3. 7 


7.2 


2.2 


1.4 


n 7 
u. / 


7 . 1 






3.9 


3.6 


4.2 


7 . 2 


12. 2 


12.8 


16.0 


10.5 


9.8 


32.1 








135. 9 




7.9 


3. 4 


29. 7 


12. 2 


27. 2 


17.7 


3.0 






7.8 


71.2 


1.3 






























0.2 




4.3 


3.0 . 




1.1 


18.7 


0.7 


3.3 


0.2 


0.7 


33. 3 


2.0 


4.4 


6.5 


0.6 










1.5 


4.4 


0.4 


0.2 


0.0 


0.5 


3.3 


0.1 


2.4 




2.8 


0 . 3 


0 . 8 


0 . 7 




0 . 1 


7 Q 


U . Z 




u . u 


1 . J 




J . 0 


1 A 7 


22. 0 


1.5 




29.5 


27.8 




2.3 


21.8 


1.4 


10.0 


0.2 


2.6 


8.5 


1.4 


10.7 


23.5 


4.1 




0.2 


0.0 




0.1 


36.7 


1.3 


0.1 


0.0 


0.8 


10.4 


2.7^ 


5.1 




2.7 




19.0 


9.1 




0.7 


73.4 


3.3 


6.3 




1.2 


40. 3 




4.7 


34. 5^ 


0.4 


0.3 


• 1.4 


1.6 


0. 7 


0.3 


24.3 


1.7 


7.8 


0.4 


1.3 




1.5 


4.4 




0.1 




0.5 


0.8 




0.8 


8.6 


0.6 


0.8 


0.2 


1.8 


10. 3 


1.2 


10.4 


3.3 


0.9 










3.3 


13.4 


1.5 


1.8 


0.8 


1.5 


4.3 


• 0.6 


7.7 


1.8 


0.1 


1.8 






0.5 




3.6 


0.5 


2.7 


1.1 


0.1 


8.9 


0.9 


5.6 


0.1 


0.0 










0.7 


13.3 


1.1 


8.9 


2.7 


1.3 


43.8 


2.4 


45.5 


2.8 














2.7 


0.2 


0.0 


0.1 


8.9 






0.7 












7.0 


0.8 


42.5 


3.4 


1.6 


0.4 


1.6 


16.6 




9.2 


12.4 



let of Title I partic- 

9. Includes Health-Medical 



2. Includes Math 



3. Includes Dental 



Includes Attendance 
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TABU 

Percentage o£ Participj 
for Programs by State' 

Arkansas Colorado Georgia Louisiana Mass. Mississippi 1 
Par. Exp. Par. Exp. Par. Exp. Par. Exp. Par. Exp. Par. Exp. ?i 



Art 










4. 


2 


1. 


2 












15.5 


1.7 


Businsss Educat.ion 


0. 


1 






0, 


9 


0. 


4 










0.9 


0.2 


0.1 


riiill'iir^l Rn T "i r^liTTK^n 1" 

\J U ^ W U t. u ^ Li L ^^11 lilC LL L> 


0 


l2 


19 . 


9 


0 


2 


0. 


1 






0 


1 


67.9^ 


0.4 


0.0 


Pno^l "f <-!h — R <^;3 H "f n t> 

l_«lt g J. J. O 1 1 LvC dU -l-Ll 


to t 


7 


43 


2 


86 


0 


56 . 


5 


76 


8 


45. 


5 




73. 8 


43. 1 


English-Speech 










0 


2 


0. 


1 


0 


5 












English-Other LA 


3 


2 


22 


4 


0 


5 


1, 


3 


1 


3 


0 


7 


19-3 


26.2 


8.7 


ESL 


























2-4 






Foreign Language 
































Home Economics 


0 


2 






0 


4 


0, 


2 










1.4 


33.5 


0.1 


Industrial Arts 


1 


0 






2 


2 


1 


2 


0 


3 


0 


2 


1.3 


2.9 


0.9 


Mathematics 


3 


5 


18 


2 


28 


9 


13 


2 


47 


7 


16 


0 


29.3 


51.1 


15.4 


Music 










6 


2 


2. 


0 












26.7 


2.6 


PE, Health, Rec. 


0 


0 






14 


8 


1 


7 










15-7 


33.7 


3.9 


Natural Science 


0 


5 






3 


2 


1 


4 










13.0 


15.5 


1.5 


Social Science 


0 


6 


7 


8 


2 


5 


1 


0 


1 


4 


0 


4 


10.9 


4.8 


0.4 


Other Voc. Ed. 


0 


3 


8 


0 


0 


9 


0 


7 


1 


1 


1 


4. 




1.0 


0.5 


Special Activities 
































for Handicapped 


8 


4 


11 


1 


2 


0 


2 


2 


6 


0 


7 


7 




2.9 


4.9 


K & PreK 


1 


3 






9 


1 


12 


8 


3 


9 


1 


6 




0.3 


0.6 


Other 


19 


6 


18 


1 


1 


8 


1 


2 


10 


6 


6 


1 


13.9 


6.4 


0.7 



/ 



Instructional 



Supportive 



Attendance 


0. 


1 






9 


.5 


0 


6 


19 


6 


1 


2 






28.4 


1 


4 


Clothing 






0 


4 










0 


0 


0 


0 






1.2 


0 


1 


Food 


0. 


l3 


3 


6 


7 


.1 






1 


4 


0 


0 


17. 


97 


8.7 


0 


2 


Guidance Counseling 


3. 


3 


25 


4 


2 


,3 


0 


2 


31 


0 


4. 


5 






45.5 


3 


1 


Health-Dental 


0. 


6 














11 


9 


1, 


4 






27.0 


1 


5 


Health- Medical 


2. 


2 


12 


7^ 










41 


5^ 


2 


2 


28. 


0^ 


49.6 


3 


6 


Library, Media Center 


0. 


0 


46 


8 


6 


.4 


0 


6 


67 


8 


3, 


7 






42.5 


3 


1 


Psychological Services 


0. 


4 


10 


0 










12 


2 


0, 


4 






22.5 


0 


5 


Social Work 


2. 


4 


3 


2 


0 


.8 


0 


2 


12 


4 


2 


1 






2.2 


0 


1 


Speech Therapy , 


1. 


i 














5 


5 


1 


5 


4. 


4 


2.5 


0 


9 


Transportation 


0. 


j» 


11 


8 






0 


75 


7 


6 


0 


3 






1.6 


0, 


6 


Special Services for 




































Handicapped 


0. 


1 


2. 


9 










5 


0 


1. 


1 






1.1 


0, 


0 


Other 


2. 


0 


14. 


8 


7 


.0 


0 


7 


16 


4 


1, 


9 


68. 


6 


16.9 


0. 


4 



1. Percentages total to more than 100 because a number of children participated in more than one program. 

2. Includes Art and Music 3. Includes Clothing and Transportation 4. May include Dental 5- Includes 
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mts and Expenditures 
for Fiscal Year 1973 



Exp. 



N. Carolina 



Pcjpn . 



Rhode 
Island 



S. Carolina S. Dakota 



VJisconsin 



Wyoming 



Par. Exp. 


Par. 


Exp . 


Par. 


Eyp . 


Par. 


Exp. 


Par. Exp. 


Par. 


Exp 




Par. Exp. 


3.0 




1. 


8 






2.6 


0.6 


0.4 


6.0 


0. 


3 


28.0 






0. 


2 






0.1 


0.0 




0.0 


0. 


0 




5.0 




1. 


6 


3.7 


0.1 


3.7 


0.5 


1.0 


18.1 


3. 


4 


37.9 


68. 2 




37. 


6 


52.1 


53.5 


63. 7 


38. 2 


/. n c 
49 . 0 


ol . 1 


35. 


7 


7 A Q 






0. 


3 




0.1 


2.7 


1.1 


2.1 


5.0 


1. 


2 




17.6 




5. 


8 


9.8 


1.2 


14.5 


7.5 


6.5 


10.4 


3. 


7 








3. 


4 


4.3 


9.3 








1.6 


1. 


0 








0. 


1 












0.1 


0.1 


0.1 






0. 


1 




0.2 


0.5 


0 . 2 




U . i 


0. 


0 








0. 


3 


1.7 


1.4 


1.7 


0.2 




0.1 


0. 


0 




9.1 




4. 


6 


16.3 


6.7 


37.8 


9.9 


22.4 


19.9 


5- 


3 


53.0 


5.7 




0. 


1 






2.9 


0.7 




4.6 


0. 


2 


27.8 


0.1 




1. 


4 


0.6 




0.9 


0.6 




10.9 


1. 


2 


42.3 


0.8 




1. 


8 




0.1 


5.7 


0.7 


0.3 


5.0 


0 . 


5 


33.3 


0.1 




1. 


9 




0.1 


3.0 


0.4 




1.0 


0. 


3 


30.0 


1.5 


• 


0. 


3 






3.0 


2.3 




0.1 


0. 


1 


5.9 


2.5 




0. 


9 


2.2 


2.5 


1.6 


4.5 


3.9 


2.5 


1. 


6 


1.0 


9.5 




7. 


3 


3.0 


3.3 


5.1 


7.3 


10.7 


19.9 


14, 


3 




1.3 




10. 


6 


27.6 


8.6 


42.6 


10.4 


25.3 


8.2 


5, 


2 


11.4 


19.0 




n 


t; 
J 




0.5 


13.1 


1.3 


6.2 


3.1 


0 


7 


21.0 


8.9 




0. 


0 




2 . 0 


3.9 


0.4 


0. 3 


3.4 


0 


8 




.7.7 




0. 


3 




0.2 


0.4 


0.7 


0.3 


-14.0 


0 


2 


18.8 


12.2 




5. 


8 


7.6 


3.7 


26.4 


2.4 


8.1 


9.3 


3 


.4 


22.9 






0. 


5 






16.4 


1.0 


1.4 


3.1 


0 


1 




39.7^ 




0. 


7 


0.6 


0.2 


54.4 


3.5 


3.9 


7.8 


1 


2 


37.2 


17.2 




3. 


7 




0.1 


10.6 


1.3 


2.4 


4.6 


0 


.3 




8.1 




0. 


9 




3.4 


12.2 


0.9 


1.2 


13.3 


3 


.4 


5.4 


29.7 




3. 


1 




3.3 


15.0 


1.3 


0.7 


8.4 


2 


.9 


5.6 






0. 


5 


0.2 




3.4 


0.5 


3.0 


, 7.4 


2 


.0 




20.5 




2. 


1 




0.8 


3.9 




4.1 


14.1 


1 


. 3 


3.4 


0.7 




0. 


4 




0.0 


7.1 


0.6 


0.5 


1.4 


3 


.6 




8.0 




0. 


3 




0.5 


31.2 


1.8 


6.0 


10.2 


6 


.3 


63.1 



The unduplicated number cf Title I participants was used to derive each program percentage. 
Food and Clothing 6. Includes Crafts, Art, and Music 7. Includes Clothing 
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Percentage o£ 
for Programs by 



Alabama Arizona Arkansas Colorado Florida 

Instructional Par. Exp. Par. Exp. Par. Exp. Par. Exp. Par. Exp. 



Art 






0.0 






0.2 


Business Education 


U . U 












Cultural Enrichment 




13. 1 


0,4 


0.2 


12.9 




bnglish-Keading 


' 45. 1 


76.8 


50.2 


38.1 


56.5 


79.1 


T7«fr1 T c? V> C T> <^ l-« 

ijngj.isn— opeecn 


U . i 


o o 
0 . 0 








6.2 


English-Other LA 


0.4 


18.5 




23.3 


27.4 


12.7 


T? CT 




13.7 








0.5 


Foreign Language 














Home Economics 


0.0 










0.3 


Industrial Arts 






0. 1 






0.3 


Mathematics 


16 . 3 


12.3 


9.2 


5.6 


22.1 


14.9 


Music 




1. J 










r ij , neaxun, ivec . 




1. 5 










Natural Science 


0.6 


1.7 






3.0 


0.6 


Social Science 


0.7 


13.8 






9.2 




Other Voc. Ed. 






2.0 


1.7 


37.8 




Special Activities 














for Handicapped 


1.5 


0.2 




12.7 


8.4 


3.9 


K & Pre-K 


9.6 




0.6 


1.0 




4.7 


Other 


4.7 


5.9 




0.7 


13.9 


5.5 


Support ive 














Attendance 


0.6 


4.2 






8.0 


5.2 


ciotning 


0. 0 


0.4 






2.0 


0.9 


r OOu 


U . 1 


3 . 9 




0. 1 


3.2 


1.8 


Guidance Counseling 


2.1 


4 . 7 


16.0 


4.6 


6.6 


7.9 


neaj. Ln—Jjencaj. 




3 . 3 


8.0 






3.4 


Heal tn— Medical 


1. / 


5.2 


19.9 


2 

3.5 


2 

11.6 


4.0 


i-iiprairy ncuia uencer 


U . J 




1. J 


0.8 


3.4 


0.1 




n n 


Z . 0 


1. '/ 




3.1 


0.5 


Social Work 


0.4 


3.8 


21.7 


3.2 


14.4 


9.0 


Speech Therapy . 


0.1 


6.2 


3.2 


2.1 




0.6 


Transportation 


2.6 


9.9 


0.3 


0.4 


0.4 


0.1 


Special Services 














for Handicapped 




0.4 


0.1 


0.2 




0.4 


Other 


17... S 


0.7 




1.7 


4.3 


12.6 



1, Percentages total to more tnan 100 because a number of children participated in more than one 

2. Includes Dental 3, Includes Clothing 



\ TABLE 2? 

irticipants and Expenditures 
\ State for Fiscal Year 1973 

1 Georgia Maine 
! Par. Exp. Par. Exp. 


Mississippi 
Par. Exp. 


Missouri 
Par. Exp. 


Nebraska 
. Par. Exp. 


South 
Carolina 
Par. Exp. 


Texas 
Par. Exp. 




2.1 


0.1 




0.0 


0.4 


0.0 






0.1 


0.1 


























11.4 


i 87.4 62.8 


99.9 


41.3 


63.3 


63.2 


59.8 


32.3 


64.0 
















9.9 


1 0.1 0.1 87.7 


20.7 


7.7 


10.2 


0.5 


10.8 


8.5 


10.0 
















3.0 












0.1 


0.0 






2.7 


0.5 




- 


0.2 


0.1 




1 34.5 16 5 38.8 


52.4 


33.1 


12.2 


16.4 


45.8 


16. 8 


16.0 




1.3 


0.1 














1.8 


0.2 




1.6 






4.0 




0.4 


0.1 


2.0 


0.1 


3.3 


1.0 


4.7 




0.1 


0.0 


1.4 




1.7 


0.7 




1 0.1 0.1 


0.3 


0.2 






1.6 


1.2 




\ 0.3 0.3 10.2 




5.8 




1.6 


10.2 


6.3 




ilO.3 15.5 




0.2 


2.9 


1.5 


5.0 


6.7 


5.5 


; 1.1 0.9 10.2 






0.9 


27.6 


3.6 


11.9 


0.2 




22.8 


1.1 






12.2 


1.1 




i 32.7 


1.9 


0.0 






3.5 


0. 7 


7.9 




0.3 


0.0 




0.4 


1.7 


0.1 


28.9 




29.1 


1.6 




26.1 


12.0 


0.1 


36.7 




29.0 


1.3 




0.1 


13.6 


1.3 


37.1 


24.9^ 1.2^ 


75.8 


3.0 


2 

2.9 


1.1 


38.9 


4.2 


66.0 


i 2.2 0.1 


6.0 


0.3 




7.8 


13.4 


0.5 




i . 8.0 


52.2 


0.9 






7.7 


1.2 " 


7.5 


0.4 0.1 8.7 


J. • u 


n 1 
u . 1 






18. 0 


1.6 




■0.3 0.1 8.0 




1.3 




0.2 


3.0 


0.5 




2.3^ 8.6 


2.0 


0.1 




0.1 


5.4 


1.0 


20.0 




0.0 


0.0 


• 




0.5 


0.2 




1 12.4 




0.5 




31.8 


27.4 


1. 7 


32.2 


rogram. The unduplicated number of Title I participants was 


used to derive each 


program percentage. 






01 












ERLC 








73 

























Table 28, in fact, shows a cdS?sistent increase in the percentage of partic- 
ipants served by reading and language arts programs ranging from 69 percent 
in 1971 to 86 percent in 1974. These figures, however, must be interpreted 
cautiously since they are based on small and different samples of states. 
The "Other" category in Tables 24, 25, 26, and 27 occasionally shows high 
percentages of both participation, and expenditures. This is due to the in- 
clusion of state program breakdowns that did not fit into any of the other 
categories used for this report- Table 28 presents a summary of these data 
by each of the four fiscal years. 

Table 29 shov7s the percentages of participants and costs in instruc- 
tional activities by state for fiscal years 1969 through 1974. There is 
wide variation among states in both participation and expenditures; however, 
it seems clear that in most states instructional activities are receiving 
attention equal to or exceeding that of supportive services . 

In summary, participation data show that there is considerable 
variation among states in number served, proportion of nonpublic participa- 
tion, percentages of students per school session and grade level, and the 
percentage of participants in each I>rogram. The total public and nonpublic 
participation figures reflect data from a sufficient number of states to 
provide a national picture of Title I participation over the last four 
fiscal years. Data are less complete for the various participation break- 
downs; however, the number of State Reports providing this information in- 
dicates that these breakdowns are feasible. Hopefully, more states will 
include these breakdowns in future reports. 
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TABLE 28 



Percentage o£ Participants and Expenditures 
Presented by Programs Across States by Fiscal Year 



Program 


FY 


71 


FY 


72 


FY 


73 


FY 


74 


Instructional 


Par. 


Exp. 


Par- 


Exp . 


Par. 


' Exp. 


Par. 


Exp. 


Art 


10. 6 


1.1 


5.9 


0. 9 


4.7 


0.9 


0.3 


0.0 


Bus iness Educat ion 


1 . 0 


0.3 


0. 2 


0. 2 


0. 2 


0.1 


0.0 


0.0 


Cultural Enrichnient 


9.5 


2.8 


10. 9 


1 . 5 


7 . 6 


0.8 


4.6 


0.0 


English-Reading 


56.8 


37. 6 


67.5 


40. 5 


69.5 


45. 3 


69.3 


47.4 


English-Speech 


3. 7 


0. 9 


1. 9 


1 . 2 


0. 8 


0. 3 


3.9 


0.0 


English-Other Lang. 


8.6 


4.0 


12.3 


5.4 


13.5 


4.4 


12.5 


5.7 


ESL 


0.3 


0. 7 


2.2 


0.1 


0.3 


0.9 


1.5 




Foreign Language 


0. 5 


0. 1 


0. 7 


0. 0 


0. 0 


0.0 






Home Economics 


0. 6 


0. 2 


0. 7 


0. 1 


0.3 


0.2 


0.0 


0.0 


Industrial Arts 


1.2 


0.6 


4.0 


0. 7 


1.2 


0.6 


0.4 


0.1 


Mathematics 


19.5 


5.6 


33.4 


10.0 


31.3 


9.6 


15.6 


19.6 


Music 


15. 9 


2. 1 


4. 3 


1 . 4 


7. 4 


1.1 


0.2 


0.0 


PE, Health, Rec. 


15.1 


2. 3 


7 . 7 


1 . 5 


10. 2 


1.3 


1.6 


0.0 


Natural Science 


14. 4 


1 . 4 


6. 4 


1 . 0 


5. 6 


1.0 


2.3 


0.3 


Social Science 


7. 9 


1.1 


4. 2 


0. 9 


3. 2 


0.8 


0. 7 


0.2 


Other Voc- Ed. 


1 . 2 


1. 0 


1. 7 


0, 9 


1 . 5 


0. 6 


1.8 


0.4 


Special Activities 


















for Handicapped 


2 . 9 


4. 7 


4 . 2 


4. 9 


2. 9 


-4.3 


2.0 


4.1 


K and PreK 


3 . 7 


5. 0 


5 . 2 


7 . 7 


5 . 4 


7 . 5 


4.0 


8.0 


Other 


19 . 1 


5. 2 


2 7.5 


6 , 4 


.13.0 


7.2 


3.0 


3.0 


Supportive 


















Attendance 


Id . 1 


-I -I 
1 . 1 


12.1 


U . O 


16 . 9 


0. 6 


5. 1 


0.5 


Clothing 


1 . 6 


0 . 2 


4 . 5 


0. 2 


2 . 7 


0.1 


6. 7 


0.1 


Food 


23 . 4 


2.7 


10. 0 


0. 8 


6. 6 


0.1 


10. 3 


0.1 


Guidance 


25.1 


3.9 


21.0 


2.7 


21.0 


3.0 


20.0 


1.4 


Health-Dental 


6.7 


0.6 


10.0 


1.0 


8.8 


0.6 


18.2 


0.5 


Health-Medical 


33,3 


2.2 


33.9 


2.0 


32.4 


1.4 


40.6 


2.3 


Library, Media Ctr 


44.2 


6.0 


17,8 


2.0 


23.3 


1.7 


2.9 


0.3 


Psychological Srvs 


14.0 


0.6 


13.2 


0.7 


9.9 


0.7 


10.6 


0.4 


Social Work 


11.2 


1.1 


10.9 


1.0 


9.9 


1.5 


5.1 


0.7 


Speech Therapy 


3.0 


0.9 


4.4 


0. 7 


2.5 


0,7 


1.6 


0.6 


Transp.ortation 


11.8 . 


1.6 


13.9 


1.1 


6.6 


0.8 


7.9 


1.5 


Spec- Srvs Handicap. 


1.3 


0.4 


0.8 


0.2 


2.2 


0.4 


0.1 


0.1 


Other 


16.0 


2.0 


25.7 


1.6 


19.0 


1.(2 


16.0 


3:3 


Number of States 


11 


11 


14 


12 


12 


8 


11 


5 



Reporting Data 



1. Percentages total to more than 100 because a number of children partic- 
ipated in more than one program. The unduplicated total number of 
Title I participants was used to derive program percentages. 
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State or 
Territory 



Alabama 

Arizona 

Arkansas 

California 

Colorado . . 

Florida 

Georgia 

Hawaii 

Idaho 

Kansas 

Kentucky 

Louisiana 

Maine 

Massachusetts 

Mississippi 

Missouri 

Montana 

Nebraska 

New Hampshire 

New Jersey 

North Carolina 

North Dakota 

Ohio 

Oklahoma 
Oregon 

Pennsylvania 

Rhode Island 

South Carolina 

South Dakota 

Tennessee 

Texas 

Utah 

Virginia 
Washington 
Wisconsin 
Wyoming 



TABLE 29 

Title I Expenditures for Instructional and Supportiv 
Participants in Instructional Activities Present, 



Fiscal Year 1969 



Fiscal Ye 



Program Costs 



Instructional Supportive 
Activities Activities 



Program Costs 



15,966,349 



% Instructional 

Partici- Instructional 
Costs pation Activities 



23,010,441 



10,094,682 


7,769,087 


56 


.5 


32 


.8 


10,683,453 


7,342, 














46,275,954 


4,783, 


4,471,924 


268,630 


94 


.3 


80 


. 1 






474,509 


75 


8 


52 


.1 






483, 770 












1,035,891 


732, 


1 n ^ A '51 Q 

XU , JO J , JXO 


D , Zoo , 9 74 


66 


8 


53 


. 3 


14,534,832 


14,168, 


3,944,361 


1,118,892 


77. 


9 


55 


8 


2,955,689 


794, 


15,652,210 


4,469,702 


77. 


8 


56 


2 






2,669,779 


494,766 


84. 


4 


56 


7 


2,809,366 




9,516,279 


2,412,212 


79. 


8 


39. 


5 


35,694,314 




7,418,000 


5,060,399 


59. 


4 


41. 


9 




4,467,685 


1,050,365 


81. 


0 


64. 


6 


4,806,205 


798, 


17,696,567 


11,170,891 


61. 


3 


45. 


8 


20,619,701 


9,398, 














1,050,000 




15,203,995 


10,019,576 


60. 


3 


45. 


3 


2,996. 


3,587,223 


358,680 


90. 


9 


63. 


4 


3,139,364 


908, 



4,356,770 



78.6 



55.4 



710,678 
15,884,760 
6,762,546 



Range 


483,770 - 
17,696,567 


268,630 - 
11,170,891 


• 56.5 - 
94.3 


32.8 - 
80.1 


710,678 - 
46,275,954 


25, 2( 
14,168,^ 


Number of States 
Reporting Data 


15 


14 


14 


14 


15 
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TABLE 29 



Instructional and Supportive Programs and Percentage of Costs and 
ructlonal Activities Presented by State and Fiscal Year 



Fiscal Year 1970 



Program Costs 



i- Instructional Supportive 
m Activities Activities 



23,010,441 

10,683,453 
46,275,954 



7,207,760 

7,342,882 
4,783,264 



% Instructional 



Fiscal Year 1971 



Program Costs 



% Instructional 

Partici- Instructional Supportive Partici- 
Costs pation Activities Activities Costs pation 



76.1 

59.3 
90.6 



61.5 

27.5 
51.7 



26,830,266 
6,231,168 
11,646,845 
65,276,686 



6,363,195 
2,177,288 
6,357,852 
7,675,977 



80.8 
74.1 
64.7 
89.5 



52.1 
60.4 
21.4 



1,035,891 
14,534,832 



732,614 
14,168,439 



58.6 



50.6 



60.9 



51.9 



2,955,689 



794,033 



78.8 



47.7 



1,657,426 

13,756,988 

23,315,393 
3,056,639 



1,007,608 62.2 

8,469,883 61.9 

5,409,133 81.2 

231,689 93.0 



43.0 
61.7 
37.6 
51.3. 
74.4 



I 2,809,366 








3,202,617 


227,588 


93.4 


85.9 


i 35,694,314 








35,003,00^ 


3,760,012 


90.3 




i 4,806,205 














47.5 


798 ,493 


85.8 


^0.5 


5,31X729 


845,529 


86.3 


75.3 


i 20,619,701 


9,398,087 


68.7 


56.1'^^ 










1,050,000 








2,431,218 


570,021 


81.0 






2,996,700 






19,205,923 


7,881,279 


70.9 


50.2 


1 3,139,364 


908,565 


77.6 


62.8 


4,310,531 


535,803 


88.9 


72.4 


710,678 


278,636 


71.8 


76.5 










1 15,884,760 


4,671,142 


77.3 


52.5 


20,701,109 


4,728,976 


81.4 


53.5 


i 6,762,546 


799,766 


89.4 


80.5 


















10,436,036 










25,265 












71.3 


i - 710,678 - 


25,265 - 


50.6 - 


27.5 - 


3,056,639 - 


227,588- 


61.9 - 


21.4 - 


46,275,954 


14,168,439 


90.6 


80.5 


65,276.686 


8,469,883 


93.4 


85.9 


i 

i 15 


14 


12 


12 


16 


15 


15 


15 



9[j 
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TABLE 29 (cont 



Fiscal Year 1972 Fiscal Year 197^ 

Program Costs % Instructional Program CQsts ; 

Instructional Supportive 
Activities Activities ; 



State or 


Instructional 


Supportive 






Partici 


Territory 


Activities 


Activities 


Costs 


pation 


A1 aTi am A 


27 861 98"^ 


A n3Q 1 n*^ 

'i' , U , XU J 


87. 


3 


ou . o 


A "f ^ On 










62 . 5 


Arkansas 


14,900 296 


3 677 766 


80. 


2 


97 1 
Z / . X 


California 












Colorado 










jx . y 


Florida 












Georgia • 
























Idaho 




1 n 1 7 •? /. 


S8 


2 


40.1 


an CI a Q 

Lv CILIO do 












Kentucky 












Louisiana 


18 184 749 


6 OQfi "^n*^ 

D , U , JU J 


7A 


q 


/. A ft 


Maine 










^ o o 

6o. 2 


Ma c: Qa p Vj 1 1 cp f- f- Q 

A' Xd OOdL^llUOC L. l-O 










^ o c 

68.5 


Ml QQl QQT TiTlT 


27 430 


H , DO J , OUZ 


85 


'■r 


50 . 2 


Miss OUT i 












Montana 












Nebraska 












Npw HaniTjcsh 1 T*«a 

i>i c w ixdiuL/ Oil Xi. c 












Npw Tptqpv 

XI w w o c 1. o c y 




.-- 








xiV^i.l_ll vidx.V^XXIid 














4 173 Tf)4 


Q8 6ns 

-7 O , Uw J 


Q7 


7 




Ohio 


38 000 34Q 


A 1 9R 6A1 


90. 


2 




vJi^Xdnouici 


1 997 1 '^7 


Q9A AQ7 


57. 


0 


oU • / 




7 3SS 4SQ 


1 ni 6nQ 

X , wX J , UU-7 


87. 


9 




Pennsvl van la 












RVinHta Tel anrl 

XvLl^iiC J-OXdIiii 


7 767 864 


361 014 

W X , W X ^ 


88. 


4 


93.2 


South Carolina 


22,988,792 


4,497,8^4 


83 


A 

D 


50.0 


South Dakota 


5,142,189 


477,894 


91. 


4 


73.5 


iiiico o tr C 


9R 7S7 38S 


S OAA 7SA 


85. 


1 


An Q 


Texas 












Utah 












Virginia 

V ^x. gXIi^d 


21,236,526 


4,178.039 


83. 


6 


SS Q 


Washington 












Wisconsin — 










57.6 


Wyoming 










76.9 


Range 


1,227,137 - 


98,605 - 


58. 


2 - 


27.1 




38,000,349 


6,096,303 


97. 


7 


93.2 


Number of States 


14 


14 


14 




17 


Reporting Data 













22,241,768 3,195,573 



38,367,652 1,171,999 



4,520,159 
4,008,518 



39,586,606 9,134,394 
3,238,853 476,567 
18,945,333 3,293,282 



20,136,565 

11,228,458 3,969,777 



3,238,853 - 476,567 "I 

39,586,606 9,134,394 



8 



90 
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TABLE 29 (cont,) 



Fiscal Year 1973 



. Progrflm Costs 



,241,768 



,,367,652. 



3,195,573 



% Tnsl-riir.f i nnal 
Partlci- 



|tructional Supportive 
tivities Act-lvlties GostR- pat-inn 



87.4 



JLa21,999 97.0 



53.1 
99.2 
83.3 



Fiscal Year 1974 



Program Costs 



Instructional Supportive 
Activities Activities 



20,689,802 
12,187,091 

36,990,743 



5,459,385 
2,419,385 

1,446,584 



% Instructional 
Particl- 
Costs patlon 



79.1 



83.4 



96.2 



34.5 
77.4 
46.5 

77.1 
73.5 
68.9 



,799,257 



1,647,278 



,586,606 
,238,853 
,945,333 



4,520,159 79.7 



4,008,518 85.0 



9,134,394 
476,567 
3,293,282 



81.3 
87.2 
85.2 



70.6 

39.2 

67.0 
51 . 2 

49.2 

90.8 
42.3 

81.7 

55.3 
93.5 
48.4 
76.2 



23,468,222 



13,820,961 



2,748,294 



4 



2,306,967 



89.5 



85.7 



36.5 

76.3 

44.8 
96.9 

62.4 



96.5 
81.7 

47.5 



1,136,565 
,228,458 



1,238,853 
,586,606 



3,969,777 



476,567 
9,134,394 

8 



73.9 



73.9 
97.0 

8 



63.6 
66.1 



39.2 
99.2 

17 



12,187,091 - 
36,990,743 



1,446,584 - 79.1 
5,459,385 96.2 



35.2 



34.5- 
96.9 

15 



i 
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Resource Allocation 



In almost all resource allocation data found in the State Title I 

' r. 

Annual Evaluation Reports, confusion existed among actual expenditures^, 
planned expenditures., estimated expenditures, and allocations. Where 
data on actual expenditures were available in the FY 71, 72, 73, and 74 
reports, these figures were extracted in preference to other data. How- 
ever, in many cases the figures were not labeled and it was impossible 
to determine whether or not they represented actual expenditures. This 
problem w^s compounded by the fact that, in some reports, carry-over 
funds from previous years were indicated separately, while in others, 
these funds may simply have been included without mention in reported 
expenditures or they may have been omitted altogether from the report. 
The same difficulty occurred with regard to funds for handicapped, de- 
linquent, and migrant children; for SEA administration; and with Part B 
and Part C funds. Where it was possible to make appropriate determina- 
tions, the expenditure data summarized in this report excluded funds 
for handicapped, delinquent, and migrant children but included all other 
funds. However, in many cases the Stat5 Reports did not include enough 
information to permit these distinctions, and it was not always clear 
exactly what the data represented. 

The resourcje allocation data presented here are only a part of 
the data that can be found in the State Title I Annual Evaluation Reports. 
Data presented in unique formats or breakdowns, data which did not clearly 
, represent the entire state program over a 12-month period, and data from 
partially legible reports were omitted. Data aggregated here were those 
appearing in a format common to several states, and those which permitted 
some comparison with the FY 69 and 70 data contained in Wargo, et al. 
(1972). 

Table 30 allows comparison of total Title I expenditures by states 
from FY 69 through FY 74. However, because, these figures arie subject 
to all the inconsistencies described above, and because they are not 
necessarily representative of all the states , caution must' be exercised 
in drawing conclusions from the data. Of the states represented in both 
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TABLIi'SO 



Title I Expenditures 
Presented by State and Fiscal Year 



State or 
Territory 


FY 69 


5Y 70 


FY 71 


FY 72 


FY 73 


FY 74 


Alabama 




40,580,250 


41,328,356 


41,687,612 


35,928,438 


43 ,419 , 741 


Arizona 


6,760,821 


7,195,996 


8,422,776 


8,648,415. 


8 134 242 


fl 9 9 1 fi 71 

0 y £.£.1. , 0 JX 


Arkansas 


22,505,629 


21,522,592 


21,948,297^ 


23,653,381^ 


22 59fi flns^ 


1 fl ^^ ^ Aflfl 1 
xo , jxx , 000 


California 


70,093,000 


63,818,240 


97,986,622 


120,909,695 


109 854 52fl 


1 Al 1 nn nnn2 
mx, xuu,uuu 


Colorado 


7,111,714 


8,773;735 


7,941,262 


10,420,000^ 


10,630,000^ 


11 nfln nnn2 


Connecticut 






10,788,070 


12,290,094 


11 538 2fiA 


1 9 nfl Q ni Q 
xz , uo 7 , uxy 


Wash. D. C. 






4,052,833 








Florida 








26 ,445 , 029 




9fl 1 Afl 09 Q 

£.0 , xoo , uzy 


Georgia 








34,171,715^ 


J ,' J J ^ , ^ J ^ 


Tfl /iA7 797! 

JO , n't / , jZ/ 


Hawaii 


2,048,479 


2 606 146 


•J 117 nQ7 




^ , yoo , zy J 




Idaho 




1,882,422 


2,665,034 


2,428,860 






Illinois 


32,173,079 




51,135,060^ 


30 299 759^ 






Indiana 












9^ 1 n«; SQi 
z J , xu J , jyx 


Kansas 


9,934,956 


10,642,16'7 


9,638,770 


9 664 169^ 


Q 7'3Q aaaI 

O , / jy , 040 




Kentucky 






35 098 405^ 








Louisiana 


28,236,296 


32,598, 848 


34 502 Q32 


797 nQ9l 


Tl 79 9 /. flQ 




Maine • 


3,350,914 


3,606,950 


5 , 454, 119 


s fin? 


AT? A7 "J 
J , 0 J J , 0 / J 


5 , 6 41 , 673 


Maryland 




10,350,353 


17,202,016^ 








Michigan 








47 onn nnn2 


SI nnn nnn2 




Minnesota 








31 434 903^ 


1 8 ififi flfi4 

X0,«400 ,00*+ 




Mississippi 


36,343,610 




28,724,526^ 


32 1 1 fi 1 57 1 


An 1 flQ "ifl?! 


77 71 n 707 1 


Missouri 






22,339,410^ 


24 605 627^ 


9 9 nfiQ IsQ?! 


91 Tflfl 7Qa1 
zx , joo , -jyH 


Montana 


2,996,462 


2,704,158 


2,993,356 


3, 297 , 707 


2 Q7fi 7^7^ 




Nebraska 


5,063,253 


5,836,290 


5,364,623^ 


6,296, 723 


5 364 623^ 




Nevada 


803,100 




863,874^ 


1 nifi 4^q1 


789, 155 ^ 




New Hampshire 








1 9fln '^Q7 


1 771 "IQ <; 1 




New Jersey 








J^ , / D± , / LL 


A^^ "JAP QA7I 

4j , , yo/ 




New Mexico 


9,935,682 


8,859,144 


8,403,380^ 


8 035 737^ 


fl Q51 I'lfil 
u , y-t±. , X J u 




New York 






193 239 909 


103 234 13q1 






North Dakota 


3,711,136 




4 188 78fi^ 




"J fl«;A QQ9I 
■J , 0 JO , 0 7 z 


A 9 1 9A7 
4 , Z jX , ZO / 


Ohio 


30, 358,091 


35 ,694 , 314 


38 763 013^ 


42 128 QQl ^ 


Al Q97 QSQI 


AA AQQ 7fl'3l 


Oklahoma' 


13, 834, 872 


17 639 029 


1 7 flTA 979 


IQ IQQ Ql/ 


1 A flQQ AQT 


1 7 9A7 fl'J9 
X / , 40 / , 0 


Oregon 


'6,057 ,342 


6 315 505 


8 1 72 842 


in 90*^ n77 1 


1 n QAfl RQfi 1 

XLF , y^Q , J i70 




Pennsylvania 


42,053,472 


48,500,000 


60,724,183 


58,216,747^ 


60,903,030^ 




Rhode Island 


3,100,856 


3,464,714 


3,677,557^ 


3,971,228^ 


4,669,684^ 




South Carolina 


29,075,524 


33,148, 316 


31,267,771^ 


32 896 138^ 


2fi flfil>^Q2 


Ifi '^'^fi 54fil 


South Dakota 


5, 314,910 


5, Km, 950 


6,136,429^ 




5 620 841 


fi fi28 42n 


Tennessee 






32, 269, 717 


31 230 697^ 






Utah 






3, 308,165 


3,593, 198 






Vermont 








l,93o, 262 


2, 141, 872 ^ 


1,761,081^ 


Virginia 


25.355.773 


30,013,202 


34.181,909^ 


33,366,919 


31,702,334 


32,894,541 


Washington 








12,272.0002 






West Virginia 






20,350,000^ 


19,974,347^ 


17,300,000^ 




Wisconsin 


13,512,957^ 


15,520,746 


15,927,775 


19,327,021 


19,402,623 




Wyoming 




1,129.222 


1,189,100^ 


1,205,406^ 


1,170,817 




BIA 






13,590,326 


12,874,580 


12,873,889 


15,155,572 


Guam 










887, 900^ 




Puerto Rico 










30,989.393 




1. Expenditure 


2. Approximate 


figure 


99 









FY 69 and FY 73, fifteen show a substantial increase in expenditures, three 
show a decrease, and six show little change. Between FY 69 and FY 74, eight 
show an increase, two a decrease, and two show little change. For most 
of the states shown, there appears to be a trend toward increasing allo- 
cations and expenditures over the years for which data are available. 

Table 31 shows estimated Title 1 per-person expenditures for states 
presenting these data or data from which these figures could be derived 
in their annual evaluation reports^ Data from FY 69 and 70 (from Wargo, 
et al. , 1972) are included in the table, but becuase of methodological 
differences in the two studies, these data are not always directly com- 
parable with the data from later years. Per-pupil expenditures for FY 69 
and 70 were calculated by the AIR reviewers, who divided total expenditures 
by total participation where these figures were reported. Per-pupil ex- 
penditures which were reported by the states were not used, and /the total 
participation figures used in the calculations included unknown amounts 
of duplication. For FY 70 through 74, per-pupil expenditures were taken 
directly from the reports when this was possible. When no per-pupil figure' 
was given, total expenditures or allocations were divided by total participa- 
tion, but only when an unduplicated couiit of total participants was avail- 
able. However, these figures, like those for FY 69 and 70, represent only 
estimates because of inconsistencies id the reporting of expenditures. 

For the states for whom data were available, the mean per-pupil ex- 
penditure has increased each year by ai|iounts ranging from $6 to $44. However, 
the range of per-pupil expenditures amd)n.g states is very wide, as can be 
seen from the table. In fiscal year 1973, the highest per-pupil expenditure 
was over 11 times greater than the lowest, and in fiscal year 1969 the 
highest was 8 times greater than the lowest. 

Table 32 shows percentages of expenditures by standard accounting 
categories for fiscal years 1966 through 1974. The table indicates an 
increase over the years in the percentage of expenditures for instruction 
and a corresponding decrease in expenditures for construction and equip- 
ment. Expenditures for fixed charges also appear to have increased com- 
pared to other expenditures. In other categories, expenditures appear 
to have changed very little over the years. As is the case with other 
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TABLE 31 



Estijnated Title I Per-pupil Expenditures Presented by 
State and Fiscal Year 



State or 



Territory 


FY 69 


FY 70 


FY 71 


FY 72_ 


FY 73 


FY 7 


Arizona 


65 


116 


169 


195 


214 


227 


Arkansas 


150 


139 


139 ' 


99 


235 




California 


279 


285 


379 


385 


320 




Loloraao 


133 


227 


210 




227 




Connec t icu t 






262 


295 


291 


— 297 


District of Columbia 






220 








Florida 








317 




343 


Georgia 








215 


302 


332 


Hawaii 


230 


331 










iaano 




49 


59 


75 


* 




Illinois 






211 


204 






Indiana 


IbO 








RNA 




Kansas 


1 c /. 

154 


154 


153 


' 187 


211 




Louisiana 


124 


206 


211 




260 




Maine 


88 






179 


RNA 


203 


Maryland 


RNA 


182 










Mis s is s ippi 


137 










255 


Missouri 








285 


270 


293 


Montana 


248 


362 


377 








Nebraska 


IJl 


75 


78 


147 


139 




Nevada 


410 




599 




378 




New Jersey 


153 






370 ' 






New Mexico 


198 


173 


155 


260 


391 




New York 






205 


237 


RNA 




North Dakota 


58 




108 




161 


177 


Ohio 


t e% t 

184 


224 


428 


472 


340 


379 


Oklahoma 


83 


RNA 


114 


163 






Oregon 


186 


183 


193 




249 




X cililojr -i- VclilJ.ci 


1 A1 
1^1 


10 J 






1G2 




Rhode Island 


lOD 


206 


231 


310 






South Carolina 


91 


106 


122 


199 


176 


135 




1 AA 


1^3 


lb J 




250 


294 


Tennessee 






144 


215 






Vermont 








139 


52 




Virginia 


171 


211 


262 


259 


235 


254 


West Virginia 






243 


294 


321 




Wisconsin 


206 


246 


275 


338 


354 




Wyoming 




69 


206 


130 


142 




BIA 








350 


580 


598 


Guam 






RNA 


RNA 


620 




Range 


58 - 


49 - - 


59- 


75 - 


52 - 


135- 




4i0 


3i2 


428 


472 . . 


620 


^98: 


Mean 


164 


184 


219 


243 


276 


291 


Number Reporting 


25 


21 


27 


26 


25 


13 



RNA= Report Not Available 
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101 
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data, these data were taken from a rather small number of states and 
are not necessarily representative of the nation. In particular, the 
FY 72 data are clearly different from those for other years because 
of the inclusion of New York in that year only. The large amounts 
expended in New York and the very deviant pattern of these expenditures 
(see Table 17) have clearly influenced the total* percentages. 

T-abi:e-s~3-37— 347-^5 , and 36 show percentages of expenditures by account 
for individual states for fiscal years 1971 through 1974. Except for 
New York, the states represented here appear to have spent their money 
in similar proportions each year, with most of the money being for 
instruction. 

Tables 24, 25, 26, and 27 show the percentages of participants and 
expenditures for programs, by state and by fiscal year. Percentages 
for individual programs vary across states; however, reading and language 
arts receive consistently high percentages of expenditures and participa- 
tion. The high percentages in the "Other" categories are the result of 
the breakdowns used here. Many states specified one or more programs 
which could not be appropriately classified into specific categories; 
these programs were therefore included in "Other" . T^ble 28 provides 
a summary of these data across states for each of the four fiscal years.. 
According to the needs assessment data which were available in the State 
Title I Annual Evaluation Reports, reading, language arts, and mathematics 
should be receiving nearly equal priority if students' critical needs 
are to be met. However, as was also pointed out in Wargo, et al. (1972), 
a surprisingly small proportion of Title I funds are devoted to mathematics, 
according to the available data. 

Table 29 shows, for fiscal years 1969 through 1974, individual 
states* expenditures on instructional and supportive activities, and 
the percentages of costs and participants in instructional activities. 
The ranges of expenditures and percentages are very wide, as can be seen 
from the table. However, instructional activities are clearly receiving 
priority in terms of expenditures. 

In summary, resource allocation data presented in the State Title I 
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Annual Evaluation Reports are neither sufficient nor consistent enough 
to form a nationally representative picture of Title I expenditures. 
However, data do appear in some reports in common formats, and in suf- 
ficient detail to indicate that the presentation of useful resource 
allocation data would be possible for most SEAs. Were other states 
to present data like those which were extracted for this report, a 
useful and representative national picture of Title I expenditures could 
be produced. 
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Impact 



To select the states to be included in the national aggregation of 
impact information, the achievement data in each State Report were examined 
to ascertain the degree to which the samples of children tested were re-, 
presentative of their state. Using the results of the data adequacy ratings 
completed in Phase I, those states whose data samples were judged to be 
"poissibly biased" or better were further examined as potential candidates 



practices that had been adopted precluded the possibility of pooling the 
data from all such states. The data from six states in 1971, six states 
in 1972, six states in 1973, and nine states in 1974 were reported in suf- 
ficiently similar formats to make a meaningful aggregation possible. Grade- 
equivalent pretest, posttest, and gain scores for these states are pre- 
sented in Tables 37, 38, 39, and 40. In some instances, the State Reports 
did not present all of tlifese measures. Where possible the measures were 
computed using the information that the reports did provide. 

Both the 1971 and 1972 aggregation^ that used this reporting format, 
reflected data from North Carolina , Rhode Island and Virginia. The 1972 and 
1973 samples had California and Rhode Island in common, 1973 and 1974 had 
California, Missouri and Nebraska in common while 1971 and 1974 included 
data from Missouri and North Dakota. The 1972 and 1974 samples shared no 
states in common. Missouri and Rhode Island were represented in both 1971 
and 1973. Rhode Island contributed data to the aggregations each year 
except in 1971. 

While the data in Tables 37, 38, 39, and 40 are not representative of 
the nation as a whole or even of a consistent group of states, some observa- 
tions can be made. The Total Sample pretest scores in Tables 37, 38, 39, 
and 40 show that by third grade the children were achieving scores below 
grade level. With each successive year the children fell farther behind 



for inclusion in the total aggregation. There were 17 states in this cate- 
gory lA\1971, 19 states in 1972, 17 states in 1973, and 11 states in 1974. 

Th'e examination of reports with adequate data samples focused on the 
reporting procedures and formats they used. The great variety of reporting 
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TABLE 37 



1971 Reading Achievement Data Expressed in Grade-equivalent 
Terms and Summarized across States with both Adequate 
Evaluation Samples and the Same Reporting Format 



State 


Grade 


N 


Pre * 


Post 


Gain 


Missouri 


1 


277 


.93 


1.49 


.56 




2 


2127 


1.55 


2.49 


.94 




3 


2458 


2.14 


2.93 


.79 




4 


6404 


2.94 


3.74 


.80 




5 


5783 


3.76 


4.52 


1. 20 




6 


5024 


4.48 


5.29 


.81 




7 


4739 


5.44 


6.40 


.96 




8 


3998 


6.29 


7.24 


.95 




9 


1068 


6.34 


7.31 


' .97 




10 


374 


6.75 


7.69 


.94 


• 


11 


137 


7. 74 


9. 23 


J. . H U 




12 


56 


8.67 


9.83 


1 16 


North 












Carolina 


1 


104 


. 72 


1 . 30 


• JO 




2 


1856 


1. 44 




. oo 




3 


1879 


2. 05 




J. . w / 




4 


1376 


3. 09 


3.92 






5 


820 


3. 88 


4 ^7 






6 


893 


4. 75 


5.25 


.50 




7 


662 


4.94 


5.46 


.52 




8 


552 


5.48 


5.94 


.46 




9 


. 338 


6.07 


7.00 


.93 




10 


99 


6.61 


7.64 


1.03 




11 ' 


6 


7,40 


8.90 


1.50 




12 


8 


5.50 


10.20 


4.70 


North 












Dakota 


1 


109 


1.46 


1.97 


.51 




2 


194 


1.90 


2.65 


.-75 




3 


243 


2.50 


3.30 


1.32 




4 


241 


3.16 


3.90 


.74 




5 


204 


4.01 


4.87 


.86 




6 


157 


4.86 


5.58 


.72 


c" 


7 


78 


5.27 


. 6.1? 


.90 




8 


46 


6.21 


7.03 


,82 
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TABLE 37 (cont.) 



State 



Grade 



Pre 



Post 



Gain 



AMG 



Rhode 
Island 



Virginia 



Wisconsin 



Total 
Sample^ 



1 . 


64 


1.4 




2,3 


.9 


2 


977 


1.4 




2.3 


.9 


3 


. 863 


2.0 




3.0 


1.0 


4 


473 


2.6 




3.6 


1.0 


5 


378 


3.3 




4.5 


1.2 


6 


308 


4.1 




5.0 


.9 


7 


277 


4.5 




6.2 


1.7 


8 


81 


5.0 




6.9 


1.9 


9 


150 


6.6 




7.8 


1.2 


1 


74 










2 


799 










3 


I/O 

142 












o on 

2o9 










5 


30^ 










b 


694 










7 


330 










o 
O 


591 


* 








n 
9 


242 










10 


608 










1 


309 










2 


765 










3 


701 










4 


497 










5 


439 • 










6 


264 










7 


279 










8 


96 










9 


32 




*- 






10 


55 











1 

2 
3 
4 
5 

- 6 
7 
8 
9 
10 
11 
12 



1.1/(554) 

1.5/(5154) 

2.1/(5443) 

3.0/(8494) 

3.8/(7185) 

4.5/(6382) 

5.3/(5756) 

6.2/(4677). 

6.3/(1556) 

6.7/(473) 

7.7/(143) 

8.3/(64) 



1.6/(554) 

2.4/(5154) 

3.0/(5443) 

3.8/(8494) 

4.5/(7185) 

5.3/ (6382) 

6.3/(5756) 

7.1/(4677) 

7.3/(1556) 

7.7/(473) 

9.2/(143) 

9.9/(64) 



.5/(554) 

.9/(5154) 

.9/(5443) 

.8/(8494) 

.7/(7185) 

.8/(6382) 

.0/(5756) 

.9/(4677) 

.0/(1556) 

.0/(473) 

.5/(143) 

.6/(64) 



1.7 
1.1 
1.4 
1.3 
1.2 
1.2 
2.6 
2.8 
2.6 

1.5 
1.2 
1.5 
1.2 
1.2 
1.0 
1.6 
1.2 
1.2 
1.5 

1.1 
1.0 
1.2 
1.1 
' 1.2 
1.0 
1.2 
.9. 
1.7 
1.5 



1.3/(447) 

1.1/(2521) 

1.3/(1706) 

1.2/(1259) 

1.2/(1121) 

1.1/(1266) 

1.8/(886) 

1.-3/(768) 

1.7/(429) 

1.5/(663) 



Rhode Island, Virginia, 



1. Sample includes Missouri, North Carolina, North Dakota, 
and Wisconsin. 

Note. Numbers in parentheses indicate the number of scores that contributed 
to each average. 



AMG= Average Monthly Gain 
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TABLE 38 



1972 Reading Achievement Data Expressed in Grade -equivalent 
Terms and Summarized across States with both Adequate 
Evaluation Sanples and the Same Reporting Format 



State 



California 



Grade « 


N 


Pre 


Post 


Gain 


1 


14466 


1-0 


1.7 


-7 


2 * 


37453 


1.5 


2.3 


.8 


3 . 


35967 


2.1 


3.0 


.9 


4 


31902 


2.8 • 


3.6 


.8 


5 


29443 


3.4 


4.2 


.8 


6 


26959 


4.0 


4.8 


.8 


7 


5663 


4.5 


5.3 


.8 


8 


4581 


5.0 


5.9 


.9 


9 


7308 


5.6 


6.8 


1.2 


10 


3486 


6.1 


7.2 


1.1 


11 


1622 


6.6 


7.9 


1.3 


12 


622 


6.6 


7.9 


1.3 



AMG 



Kansas 



2 
3 
4 
5 
6 
7 
8 



1766 
1658 
1621 
1618 
1348 
1407 
723 



1.67 
2.40 
3.04 
3.68 
4.33 
5.29 
5.66 



2.56 
3.20 
3.91 
4.58 
5.30 
6.28 
6.67 



.89 
.'80 
.87 
.90 
.97 
.99 
1.01 



1.11 
1.03 
1.09 
1.12 
1.21 
1.24 
1-26 



North 
Carolina 



1 
2 
3 
4 
5 
6 
7 
8 



556 
4025 
4123 
2827 
2654 
2604 
2028 
1245 
1124 



.74 
1.57 
2.07 
2.64 
3.32' 
3.98 
4.69 
5.40 
5.78 



1.29 
2.25 
2.81 
3.52 
4.12 
4.69 
5.43 
5.88 
6.49 



.55 
.68 
.74 
.88 
.«0 
.71 
.74 
.48 
.71 



Rhode 
Island 



1 
2 
3 
4 
5 
6 
7 
8 
9 
10 
11 
12 



67 
1048 
967 
500 
246 
271 
394 
152 
236 
19 
12 
7 



1.39 
1.49 
2.02 
2. .67 
3.25 
3.85 
4.33 
5.16 
6.04 
4.85 
5.45 
4.83 



1.98 
2.46 
3.14 
3.53 
3.97 
4.85 
5.62 
6.29 
7.41 
6.13 
6.68 
5.89 



.59 
.97 
1.12 
.86 
.72 
1.00 
1.29 
1.13 
1.37 
1.28 
1.23 
1.06 



1.08 
1.27 
1.48 
1.16 
.92 
1.19 
1.47 
1.55 
2.41 
2.10 
2.14 
1.52 
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TABLE 38 (cont.) 



State 



Grade 



N 



Pre 



Post 



Gain 



AMG 



Utah 



1 
2 
3 
4 
5 
6 
7 
8 
9 
10 
11 
12 



1616 
891 
904 
683 
684 
689 
729 
378 
201 
341 
275 
207 



.84/(1581) 
1-10/(860) 
1-10/(861) 
-94/(627) 
-95/(632) 
-99/(633) 
1-15/(710) 
1-21/(367) 
-92/(201) 
1-10/(341) 
-65/(275) 
-76/(207) 



1-0/(1616) 
1-3/(891) 
1-2/(904) 
1-0/(683) 
1-0/(684) 
1-0/(689) 
1-4/(729) 
1-5/(378) 
1-2/(201) 
1-5/(341) 
-8/(275) 
1-0/(207) 



Virginia 



1 
2 
3 
4 
5 
6 
7 
8 
9 
10 



1327 
322 
839 
352 
305 
371 
857 
683 
135 
180 



-9 
1-7 

-9 
1-0 
1-1 
1-1 
1-0 
1-4 
1-5 
1-2 



Total 
Sample^ 



1 
2 
3 
4 
5 
6 
7 
8 
9 
10 
11 
12 



1-0/(15089) 

1- 5/(44292) 

2- 1/(42715) 

2- 8/(36850) 

3- 4/(33961) 

4- 0/(31182) 

4- 7/(9492) 

5- 2/(6701) 

5- 6/(8668) 

6- 1/(3505) 
6-6/(1634) 
6-6/(629) 



1- 7/(15089) 

2- 3/(44292) 

3- 0/(42715) 

3- 6/(36850) 

4- 2/(33961) 

4- 8/(31182) 

5- 5/(9492) 

6- 0/(6701) 

6- 8/(8668) 

7- 2/(3505) 
7-9/(1634) 
7-9/(629) 



-7/(16675) 
-8/(45152) 
-9/(43576) 
-8/(37477) 
-8/(34593) 
-8/(31815) 
-8/(10202) 
-8/(7068) 
1-2/X8869) 
1-1/(3846) 
1-2/(1909) 
1-2/(836) 



1-0/(3010) 
1-2/(4037) 
1-1/(4368) 
1-1/(3156) 
1-1/(2853)' 
1-1/(2679) 
1-2/(3387) 
1.4/(1936) 
1-8/(572) 
1-4/(540) 
-9/(287) 
1-0/(214) 



1- Sample includes: California, Kansas, North Carolina, Rhode Island, Utah, 
and- Virginia - 

Note. Numbers in parentheses indicate the number of scores contributing 
to each average - 

AMG= Average Monthly Gain 
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TABLE 39 



1973 Reading Achievement Data Expressed in Grade -equivalent 
Terms and Summarized across States with both Adequate 
Evaluation Samples and the Same Reporting Format 



State 



California 



Missouri 



Nebraska^ 



Grade 


N 


Pre 


Post 


Gain 


1 


12448 ■ 


1.0 


1.7 


.7 


2 


43213 


1.5 


2.3 


.8 


3 


■ 43614 


2.1 


2.9 


.8 


4 


33569 


2.8 


3.6 


.8 


5 


32716 


3.5 


4.2 


.7 


6 


29500 


4.1 


4.9 


.8 


7 


5498 


4.5 


5.3 


.8 


8 


3012 


5.0 


6.0 ' 


1.0 


9 


8439 


5.7 


6.8 


1.1 


• 10 


3739 


6.2 


7.1 


.9 


11 


1393 


6.4 


7.4 


i.a 


12 


589 


6.9 


7.8 


.9 


1 


323 


1.1 


1.7 


.6 


2 


4382 


1.8 


2.5 


.7 


3 


4251 


2.4 


3.6 


1.2 


4 


4237 


3.0 


4.3 


1.3 


5 


4268 


4.1 


5.3 


1.2 


6 


3367 


4.5 


5.8 


1.3 


7 


2465 


4.9 


6.2 


1.3 


8 


2254 


5.4 


6.5 


1.1 


9 


525 


5.6 


6.9 


1.3 


10 


163 


5.7 


7.2 


1.5 


11 


112 


6.4 


7.5 


1.1 


12 


43 


7.4 


8.5 


1.1 


1 


278 






.63 


2 


1617 






1.07 


3 


2148 






1.07 


4 


2584 






1.01 


5 


2564 






1.08 


6 


2281 






1.15 


7 


1511 






1.25 


8 


1145 






1.11 


9 


515 






.94 


10 


306 






1.08 


11 


160 






.93 


12 


164 






1.02 



AMG 



ERIC 
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TABLE 59 (cont.) 



Stat^ Grade ^ N Pre Post Gain ^ AMG 

Rhode 
Island 



South 
Dakota^ 



S 



1 


48 


1.2 


1.7 


.5 


. 6 


2 


721 


1.5 


2.5 


1.0 


1.4 


3 


792 


1.9 


3.1 


1.2 


1 s 


4 


519 


2.6 


3.7 


. 1.1 


X « J 


5 


287 


3.4 


4.6 


1.2 




6 


224 


4.2 


5.5 


1.3 


1 7 

X • / 


7 


543 


4.6 


6.0 


1.4 


1 7 

X • / 


8 


219 


4.9 


6.3 


1.4 




9 


139 


5.9 


7.2 


1.3 


1.9 


10 


67 


5.2 


6.6 


1.4 


2.3 


11 


17 


5.6 


6.2 


.6 


.9 


12 


5 


5.5 


7.0 


1.5 


2.6 


K 


352 








1.04 


1 


708 








.88 


2 


1233 








.98 


3 


1425 








.91 


4 


1459 








.87 


5 


1282 








.87 


6 


1142 








.89 


7 


633 








.87 


8' 


481 








.86 


9 


48 








.81 


I'O 


10 








.99 



Verunont^ 1 57 1.07 1.62 .55 .75 

2 411 1.65 2.47 .82 1.10 

3 443 2.09 3.08 .99 1.27 
^ 352 2.69 3.28 .59 .84 

5 196 . 3.29 3.82 .53 .80 

6 203 4.05 5.05 1.00 1.37 

7 82 4.70 5.43 .73 " 1.03 

8 41 4.78 5.31 .53 .78 
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TABLE 39 (cont.) 



State 



Grade 



Pre 



Post 



Gain 



AMG 



Total 
Sample' 



1 
2 
3 
4 
5 
6 
7 
8 
9 
10 
11 
12 



1.0/(12596) 

1.5/(48727) 

2.1/(49100) 

2.8/(38677) 

3.6/(37467) 

4.1/(33294) 

4.6/(8588) 

5.2/(5526) 

5.7/(9103) 

6.2/(3969) 

6.4/(1522) 

6.9/(637) 



1.7/(12596) 

2.3/(48727) 

2.8/(49100) 

3.7/(38677) 

4.3/(37467) 

5.0/(33294) 

5.6/^8588) 

6.2/(5526) 

6.8/(9103) 

7.1/(3969) 

7.4/(1522) 

7.8/(637) 



.7/(12874) 
.8/(50344) 
.7/(51248) 
.9/(41251) 
.7/(40031) 
.9/(35575) 
1.0/(10099) 
1.0/(6671) 
1.1/(3618) 
.9/(4275) 
1.0/(1682) • 
.9/(801) 



.9/(813) 
1.1/(2365) 
1.1/(2668) 
1.0/(2330) 

.9/(1765) 
1.1/(1569) 
1.5/(i060) 
1.2/(741) 
1.6/(137) 
2.1/(77) 



1. The total number of children contributing gain scores is given. However, 
the grade level N's were calculated from percentage data listed in another 



2. The AMG numbers were assumed to be average monthly gains although they 
were labeled gain scores. 

a 

3. The AMG numbers were calculated by RMC based on the testing dates cited 
in the report. 

4. The sample includes California, Missouri, Nebraska, Rhode Island, South 
Dakota, and Vermont. 

Note. Numbers in parentheses indicate the number of scores contributing 
to each average, 

AMG= Average Monthly Gain 
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TABLE 40 



A 

1974 Reading Achievement Data Expressed in Grade -equivalent 
Terms and Summarized across States with both Adequate 
Evaluation Samples and the Same Reporting Format 



State 


Grade 


N 


Pre 


Post 




AMP 


Alabama 


1 


10272 




1 51 

X • ^x 








2 


14455 


1.14 


2.16 








3 


13424 


1.80 


2.80 


1 no 






4 


10623 


2.40 


3.29 


89 






5 


9406 


3.00 


4.04 


X . 






6 


8402 


3.68 


4.63 ' 








7 


3249 


4.26 


5.45 


1 1 9 

X . X ^ 






8 


1875 


4.73 


5.80 


1 07 




BIA 


1 


254 






.8 






9 


Aft? 






1. U 








69*^ 






1 . U 






L 


J / J 






• 7 








D 1 1 






. 9 






o 


AR9 






.9 






7 


jyy 






1.2 






Q 
O 








1.2 






Q 


A An 






.9 






1 n 








1.1 






1 1 


901 






1.2 






1 9 

xz 


X 






.9 




California 


1 


19668 


1.0 


1.8 


.8 


1.1 




2 


58539 


1 7 
X . / 


9 A 


7 


X . U 




3 






J 4 u 


7 


1 . U 




4 


45203 


2.8 


3.7 


.9 


1.3 




5 


42371 


3.6 


4.3 


.7 


1.0 




6 


41489 


4.2 


5.0 


.8 


1.1 




7 


13881 


4.6 


5.3 


.7 


1.0 




8 


7156 


5.1 


6.0 


.9 


1.3 




9 


12135 


5.9 


6.9 


1.0 


1.4 




10 


4293 


6.4 


7.2 


.8 


1.1 




11 


1462 


6.3 


7.2 


.9 


1.3- 




12 


605 


6.5 




.9 


- 1^3 


Guam^ 


1 


61 


2.JL " 


3.1- 


. 1.0 


1.3 




2 


-70 


"2.2 


3.6 


1.4 


1.0 



liT 
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TABLE 40 Ccont.) 



State Grade N Pre Post Gain AMG 

Kentucky 1 1 369 .71 .91 

.81 .91 

1.04 1.13 

.85 .85 

.77 .77 

.49 .49 

.87 .99 

.84 1.40 

1.30 1.30 



Grade 


N 


1 


369 


2 


245 


3 


167 


4 


60 


5 


20 


6 


14 


7 


4 


8 


5 


9 


1 



Missouri 1 560 .83 1.75 .92 

2 4035 1.37 2.51 1.14 

3 5043 2.04 3.29 1.25 

4 5497 2.60 3.53 .93 

5 5445 3.34 4.42 1.08 

6 ' 4540 4.14 5.19 1.05 

7 3468 4.58 5.46 .88 

8 2753 5.14 5.98 .84 

9 721 5.55 6.87 1.32 

10 352 5.88 7.22 1.34 

11 102 7.10 8.87 1.77 



Nebraska^ 1 396 .72 

2 2030 • . ' 1.04 

3 2289 1.01 

4 2402 1.00 

5 2481 1.09 

6 2104 l.a3 

7 1471 1.22 

8 924 .94 

9 . 419 .80 

10 183 .83 

11 151 .72. 

12 77 .75 

North 

Dakota 2 930 1.53 2.60 1.07 1.33 

3 1041 2.25 3.26 1.01 1.23 

4 1091 2.99 3.99 1.00 1.21 

5 956 3.68 4.70 1.02 1.26 

6 829 4.31 5.35 1.04 1.29 

7 434 4.77 5.74 .97 1.20 

8 387 5.37 6.48 1.11 1.45 

9 97 5.02 5.99 .97 1.23 

10 65 6.52 7.72 1.20 1.33 

11 2 4.55 4.50 .05 .00 

. 98 
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TABLE 40 (cont.) 



State 


Grade 


N 


Pre 


Post 


Gain 


AMG 


Texas 


2 


7165 








1.04 




3 


8149 








.96 




4 


8164 








1.02 




5 


8128 








.99 




6 


5105 








.85 




7 


2815 








.93 




8 


1243 








.75 




9 


1004 








1.05 




10 


412 








1.13 




11 


193 








.81 




12 


145 








1.01 


Total 














3 

Sample 


1 




l;0/(20289) 


1.8/(20289) 


.8/(20912) 


1.1/(20494) 




2 




1.6/(78029) 


2.4/(78029) 


.8/(78761) 


1.0/(68979) 




3 




2. 2/(76891) 


3.0/(76891) 


.8/(77681) 


1.0/(69029) 




A 




2.6/(64816) 


3.5/(64816) 


.9/(65451) 


1.2/(56920) 




5 




3.5/(58178) 


4.3/(58178) 


.8/(58775) 


1.0/(53956) 




6 




4.0/(57364) 


4.8/(57364) 


.8/(57830) 


1.1/(49541) 




7 




4.6/(21032) 


5.4/(21032) 


.8/(21435) 


1.0/(18605) 




8 




5.1/(12171) 


6.0/(12171) 


.9/(12629) 


1.2/( 9715) 




9 




5.9/(12953) 


6.9/(12953) 


1.0/(13594) 


1.4/(13656) 




10 




6.4/( 4710) 


7.2/( 4710) 


.9/( 4911) 


l.l/( 4953) 




11 




6.4/( 1566) 


7.3/( 1566) 


1.0/( 1767) 


1.2/( 1808) 




12 




6.5/( 605) 


7.4/( 605) 


.9/( 133) 


1.2/( 827) 



1. AMGs were calculated by RMC. 

2. The total number of children contributing gain scores is give. However, 
the grade level N's were calculated from percentage data listed in another 
table. 



3. Sample includes Alabama, the Bureau of Indian Affairs, California, Guam, 
Kentucky, Missouri, Nebraska, North Dakota, and Texas. 

Note. Numbers in parentheses indicate the number of scores that contributed 
to each average. 

AM&= Average Monthly Gain 
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so that by seventh grade the 1971 sample tested two years below grade 
level and the 1972, 1973, and 1974 samples tested almost three years 
below grade level. 

Clearly inconsistent with the pretest indications of achievement 
status are the data presented in the average monthly gain (AMG) columns 
which show that, with few exceptions, the children were making a month gain 
or more for each month of instruction. Theoretically, if disadvantaged 
children were able to maintain this rate of *growth, they would no longer 
fall farther and farther behind their more advantaged counterparts. Un- 
fortunately, arguments presented by Tallraadge and Horst (1974) and Horst, 
Tallmadge, and Wood (1975) indicate that average monthly gain data are 
highly suspect when computed for a pre- to posttest interval of less 
than twelve months. Thus it appears likely that actual gains were smaller 
than those reported, and that the inconsistency between status indications 
and gains resulted from systematic biases inherent in grade-equivalent 
scores and was really no more than an artifact of measurement. 

By referring again to the Total Sample sections in Tables 37, 38, 
39, and 40, pretest, posttest, and gain scores can be compared across 
the four years. The scores for grades one through four are roughly , com- 
parable in 1971, 1972, 1973, and 1974. After fourth grade the 1971 data 
begin to diverge increasingly from the data of the following years. One 
explanation for this is that the 1971 data were less stable because of the 
smaller sample size. This instability is reflected in the differences in 
pretest, posttest, and gain scores from the other years. A second ex- 
planation, and one which is easily verifiable, is that the similarity 
of the 1972, 1973, and 1974 data was a result of the fact that California 
contributed approximately 70%, 85%, ai^d 95% of the total sample- size re- 
spectively in these three years. The Total Sample data were thus heavily 
influenced by California, and the different picture reflected by the 1971 
data was largely due to the absence of California from that aggregation. . 

The comparability problems which exist with respect to the FY 71, 
72, 73, and 74 impact data aggregations would be still more severe if an 
attempt were made to include results from ¥Y 69 and 70. While impact data 
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for those years were aggregated by Wargo, et al (1972), the aggrega- 
tions were for grades two. four, and six only. More important, data 
were pooled across t^pes of programs and subject matters rather than 
restricting the aggregations to reading programs and reading achievement 
tests as was done here. For these reasons, no attempt has been made to 
look for trends across the six-year period. 

Three 1972, four 1973, and six 1974 State Reports shared a second 
reporting format making another aggregation possible. The number and 
percent of children who had average monthly gains within each of four 
specified ranges are shown in Tables 41, 42, and 43 for these states. 
Again, wherever possible, missing values were computed from the existing 
data. These aggregations contain many of the same deficiencies mentioned 
previously. In addition, the states using this format did not employ 
identical AMG ranges. Fortunately, the ranges chosen were sufficiently 
similar so that aggregation was still possible. Two further points 
should be noted. First, the size of the Total Samples of children tested 
differed each year ranging from 188,188 in 1974 to 261,382 in 1973. 
Second, the 1972 and 1973 aggregations have only the state of California in 
common; both the 1973 and 1974 aggregations have data from Florida, Missouri, 
and* Nebraska; and both the 1972 and 1974 samples include data from North 
Dakota. No single state contributed data to each of the three yearly 
aggregations . 

Clearly, the data presented in Tables 37 through 43 do not consti- 
tute an .adequate basis for assessing the national impact of the Title I 
program. It is not even possible to draw meaningful inferences about 
national or regional trends from year to year since different samples of 
states are represented in each yearly aggregation. 

While the type of aggregation presented in Table 44 shows a significant 
difference between years, the Average Monthly Gains computed for the 1972, 
1973, and 1974 Total Samples (see Tables 38, 39, ahd 40) , -show essentially 
no differences between the three years. This apparent inconsistency between 
the two sets of data could be due to any one of a variety of possible causes 
such as differences in the composition of the two samples, skewness of the 
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TABLE 41 



Number and Percent of Children who Tested within Specified Averagi 
Monthly Gain Score Ranges in Reading for 1972: States with 
Adequate Sampling Procedures and the Same Reporting Format. 



California 







>1. 


4 


1.4-1.0 


.9- 


.7 


<.7 




Grade 


N 


nl 


% 


n 


% 


n 


% 


n 


% 


1 


14466 


216 


1 


2189 


15 


5474 


38 


6547 


45 


2 


37453 


2611 


7 


9799 


26 


10372 


28 


14669 


39 


3 


35967 - 


2461 


7 


12205 


34 


11056 


31 


10247 


28 


4 


31902 


2758 


9 


8436 


26 


9134 


29 


11574 


36 


5 


29443 


1844 


6 


5589 


19 


9822 


33 


11588 


39 


6 


26959 


1790 


7 


7021 


26 


9806 


36 


8341 


31 


7 


5663 


961 


17 


977 


17 


1173 


21 


2552 


45 


8 • 


4581 


617 


13 


680 


15 


3090 


67 


193 


4 


9 


7308 


2124 


29 


2905 


40 


1320 


18 


: 958 


13 


10 


3486 


832 


24 


101-8 


29 


610 


17 


1026 


29 


11 


1622 


385 


24 


516 


32 


272 


17 


448 


28 


12 


622 


224 


36 


170 


27 


82 


13 


, 146 


23 



New Jersey 



Grade 


N 


>1.50 

n % 


1.50-1 
n 


.01 

% 


1.00- 
n 


.*71 

% 


<.71 

n % 


1 


1908 


488 


26 


— ^ , 

184 


10 


191 


10 


1045 


55 


2 


5112 


1247 


24 


983 


19 


785 


15 


2097 


41 


3 


4947 


1531 


31 


853 


17 


697 


14 


1186 


24 


4 


3645 


951 


26 


624 


17 


530 


15 


1534 


42 


5 


2733 


783 


29 


442 


16 


330 


12 


1178 


43 


6 


1816 


599 


33 


283 


16 


204 


11 


730 


41 


7 


1098 


399 


36 


143 


13 


106 


8 


450 


55 


8 


819 


286 


35 


103 


13 


64 


8 


366 


45 


9 


1280 


581 


45 


171 


13 


96 


8 


432 


34 


10 


235 


131 


56 


24 


10 


16 


7 


64 


27 


11 


12-3 


70 


57 


12 


10 


5 


4 


36 


29 


12 


66 


37 


56 


7 


11 


3 


5 


19 


29 



..J,22 

102 



TABLE 41 (cont.) 



North 
Dakota 







>1.50 


1.50- 


1. 01 




-.71 


. <.7l' 


Grade 


N 


n 


7. 


n 


7, 


n 


% 


n 


% 


1 


436 


66 


15 


59 


14 


95 


22 


216 


50 


2 


956 


206 


22 


216 


23 


167 


17 


367 


38 


3 


1237 


252 


20 


253 


20 


246 


20 


486 


39 


4 


1158 


184 


16 


205 


18 


245 


21 ° 


523 


45 


5 


1128 


• 167 


15 


221 


20. 


215 


19 


525 


47 


6 


927 


161 


17 


150 


' 16 


163 


/ 18 


453 


49 


7 


495 


88 


18 


72 


15 


107 


22 


228 


46 


. 8 


259 


57 


22 


59 


23 


47 


18 


96 


37 


9 


42 


7 


17 


5 


12 


7 


17 


23 


55 



Total Sample^ 



Grade 



N 



>1.50' 



7. 



1.50-1.01 

n % 



1.00-.71 

n % 



<.71 

n % 



1 
2 
3 
4 

5 
6 
7 
8 
9 
10 
11 
12 



16810 
43521 
42151 
36705 
33304 
29702 
7256 
5659 
8630 
3721 
1745 
688 



770 
4064 
4153 
4867 
3755 
-3316 
1448 
960 
2712 
963 
455 
261 



5 
9 
10 
13 
11 
11 
• 20 
17 
31 
26 
26 
38 



2432 
J.0998 
1.3311 
9265 
6252 
7454 
1192 
843 
3082 
1042 
528 
177 



14 
25 
32 
25 
19 
25 
16 
15 
36 
28 
30 
26 



5760 
11324 
11999 
9910 
10367 
10173 
1386 
3201 
1432 
626 
277 
85 



34 
26 
28 
27 
31 
34 
19 
57 
16 
17 
16 
12 



7808 
17133 
11919 
13631 
13291 
9524 
3230 
655 
.1413 
1090 
484 
165 



46 
39 
28 
37 
40 
32 
45 
12 
16 
29 
28 
24 



1. The n's may not siim to the reported N and the percentages may 
not sum to 100% due to rounding error. 

2. Sample includes California, New Jersey, and North Dakota. 

3. All the states did not separate scores into the same ranges 
of grade-equivalent scores. However, the ranges overlap^ 
sufficiently to permit their aggregation in the total 
sample. ' 
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table' 4 i2 



Number and Percent of Children who Tested within Specified Average 
Monthly Gain Score Ranges in Reading for 1973: States with 
Adequate Sampling Procedures and the Same Reporting Format., 



California 



Grade 


N 


>1,. 
n 


5 


1.4-1, 

n 


0 

% 


.9-, 7 ■ 

n % 


<,7 

n % 


1 


14519 


• 396 


3 


1319 


9 


4294 


30 


8510 


59 


2 


43434 


1571 


4 


11534 


27 


13834 


32 


16496 


38 


3 


43941 


2673 


6 


10573 


24 


15339 


35 


15357 


35 


4 


33841 


2231 


7 


7700 


23 


9928 


29 


13982 


41 


5 


33028 


1595 


5 


•6669 


20 


10143 


31 


14917 


45 


6 


29755 


1722 


6 


5738 


19 


10850 


36 


11444 


38 


7 


5582 


317 


6 


949 


17 


917 


16 


3399 


61 


8 


3046 


264 


9 


762 


25 


1389 


46 


632 


21 


9 


8556 


2343 


27 


1820 


21 


1857 


22 


2536 


30 


10 


3859 


409 


11 


1368 


35 


1369 


35 


712 


18 


11 


1448 


390 


27 


409 


28 


304 


21 


•345 


24 


12 


627 


117 


19 


292 


47 


. 25 


4 


193 


31 



Florida 







>1,50 


1,50-1.01 


1.00- 


.71 


<.71 




Grade 


N 


n 


% 


n 


% 


n 


% 


n ■ 


% 


1 


767 


2 


.2 


139 


18 


452 


59 


174 


23 


2 


9152 


1147 


13 


1312 


14 


3578 


39 


3115 


34 


3 


8041 


1474 


18 


1726 


21 


1169 


15 


3672 


46 


4 


.5583 


256 


5 


1548 


28 


1159 


21 


2620 


47 


5' 


4627 


296 


6 


1195' 


26 


981 


21 


2232 


48 


6 


4149 


614 


15 


792 


19 


390 


9 


2353 


57 


7 


3271 


313 


10 


683 


21 


1361 


42 


906 


28 


8 


1997 


179 


9 


218 


11 


787 


39 


813 


41 


9 


1759 


51 


3 


126 


7 


316 


18 


1266 


72 


10 


1367 


207 


15 


255 


19 


204 


15 


701 


51 


11 . 


245 


9 


4 


5 


2 


220 


90 


5 


2 


12 


26 


1 


4 


6 


23 


0 


0 


19 


73 
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TABLE 42 (cont.) 
Missouri 







>1. 


50 


1.50- 


1.01 


1.00- 


.71 


<. 


71 


Grade 


N 


n 


% . 




% 


n 


% 


n 


% 


1 - 


323 


29 


9 


52 


16 


95 


29 


147 


45 


2 


4382 


750 


17 


1014 


23 


891 


20 


1727 


39 


3 


4251 


738 


17 


894 


21 


1262 


30 


1309 


31 


4 


4237 


1147 


27 


947 


22 


674 


16 


1447 


34 


5 


4268 


1261 


30 


992 


23 


815 


19 


1199 


28 


6 


3367 


1108 


33 


691 


21 


492 


15 - 


1234 


37 


7 


2465 


638 


26 


509 


21 


356 


14 


961 


39 


8 


2254 


^ 700 


31 


455 


20 


390 


17 


708 


31 


9 


525 


211 


40 


73 


14 


70 


13 


171 


33 


10 


163 


76 


47 


20 


12 


14 


9 


52 


. 32 


11 


112 


34 


31 


18 


16 


19 


17 


41 


37 


12 - 


43 


13 


31 


10 


22 


7 


17 


13 


30 



Nebraska 







>1. 


50 


1.50- 


1.01 


1.00 


-.71 


<.71 


Grade 




n 


% 


n 


% 


:: n 


% 


n 


" % 


1 


313 


19 


6 


38 


12 


56 


18 


200 


64 


2 


1819 


398 


22 


454 


25 


340 


19 


627 


34 


3 


2416 


578 


24 


512 


21 


430 


18 


896 


37 


4 . 


" 2907 


669 


23 


614 


21 


414 


14 


1210 


42 


5 


2884 


824 


29 


580 


20 


398 


14 


1082 


38 


6 


2566 


759 


30 


445 


17 


355 


14 


1007 


39 


7 


1700 


597 


35 


249 


15 


220 


13 


634 


37 


8 


1288 


406 


32 


196 


15 


139 


« 11 


547 


42 


9 


579 


168 


29 


106 


18 


96 


1>- 


-,209 


36 


10 


344 


92 


27 


57 


'l7 


32 


9 


163 


47 


11 


180 


50 


28 


27 


15 


21 


12 


82 


46 


12 


184 


50 


27 


27 


15 


14 


8 


93 


51 
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TABLE 42 (cont.) 



Total Sample 







>1.502 


1. 50-1 


01 


1 . 00- 


• 71 


< . 71 




rade 


N 


Li 


/o 


n 


% 


n 


% 


n 


% 


1 


15922 


446 


3 


1548 


10 


4897 


31' 


9031 


57 


2 


58787 


3866 


7 


14314 


24 


18643 


32 


21965 


37. 


3 


58649 


5463 


9 


13705 


23 


.18198 


31 


21234 


36 


4 


46568 


4303 


9 


10809 


23 


12175 


26 


19259 


41 


5 


44807 


3976 


9 


■ 9436 


21 


12337 


28 


19430 


43 


6 


39837 


4203 


11 


7666 


19 


12087 


30 


16038 


40 


7 


13018 


1865 


14 


2390 


18 


2854 


22 


5900 


45 


8 


8585 


1549 


18 


1631 


19 


2705 


32 


2700 


31 


9 


11419 


2773 


24 


2125 


19 


2339 


,20 


4182 


37 


LO 


5733 


784 


14 


1700 


30 


1619 


28 


1628 


28 


LI 


19?5 


483 


24 


459 


23 


564 


28 


473 


24 


L2 


880 


181 


21 


335 


38 


46 


5 


318 


36 



1 The total sample includes California, Florida, Missouri, and Nebras 

2 The sample states did not use the same ranges of grade-equivalent 
scores but the ranges overlap sufficiently to permit aggregation. 
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TABLE 43 

•Number and Percent of Children who Tested within Specified Average 
Monthly Gain Score Ranges in Reading for 1974: States with 
Adequate Sampling Procedures and the Same Reporting Format. 



Alabama 







> 1. 


4 


1.4-1.0 


.9-. 


7 


< .7 




Grade 




n 


% 


n 


% 


n 


% 


n 


% 


N 














1 


9331 


2972 


32 


2653 


28 


1998 


21 


1708 


18 


2 


14068 


3366 


24 


4001 


28 


2758 


20 


3943 


28 


3 


12936 


3166 


24 


3316 


26 


2415 


19 


4039 


31 


4 


10483 


2445 


23 


2332 


22 


2111 


20 


3595 


34 


5 


8895 


2546 


29 


1968 


22 


1681 


19 


2700 


30 


6 


8048 


2129 


26 


1720 


21 


1481 


18 


2712 


34 


7 


3435 


884 


26 


693 


20 


754 


22 


1104 


32 


8 


2014 


565 


28 . 


479 


24 


441 


22 


529 


26 


9 


962 


1 288 


30 


183 


19 


184 


19 


307 


32 


10 


115 


56 


49 


24 


21 


12 


10 


23 


20 


11 


101 


1 44 


44 


9 


9 


13 


13 


35 


35 


12 


: 99 


33 1 33 


20 


20 


16 


16 


1 30 


30 

■ 



Arizona 



Grade 

— H 


N 


> 1. 
n 


4 

% 


1.4-1.0 

n % 


.9-. 

n 


5 i 
% i 


< .5 

n % 




2704 


771 


29 


551 


20 


733 


27 i 


649 


24 




2832 


780 


28 


632 


22 


691 


24 


729 


26 


4 


2299 


634 


28 


481 


21 


487 


21 : 


697 


30 


5 


2322 


761 


33 


461 


20 


462 


20 


638 


27 


6 


1982 


699 


35 


387 


20 


36r3 


18 


533 


27 


7 


1459 


559 


38 


260 


18 


245 


17 


395 


27 


8 


1250 


553 


44 


164 


13 


181 


14 


352 


28 


9 


2150 


957 


45 


332 


15 


302 


14 


559 


. 26 


10 


879 


400 


46 


142 


16 


142 


16 


195 


• 22 


11 


303 


114 


38 


44 


15 


67 


22 


78 


26 


- 12 


121 


57 


47 


15 


12 


23 


19 


26 


21 
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TABLE 43 (cont.) 



Florida 







> 1 


.49 


1.49- 


■1.00 


.99- 


.50 


< .50 


Grade 


N 


n 


% 


n 


% 


n 


% 


n 


% 


1 


2000 


12 


01 


83 


04 


1527 


76 


378 


19 


2 


10237 


491 


• 5 


1643 


16 


6905 


67 


1198 


12 


3 


9179 


803 


9 


1356 


15 


6441 


70 


579 


6 


4 


6458 


636 


10 


1541 


24 


3712 


57 


569 


9 




7355 


1036 


14 


3229 


44 


2436 


33 


654 


9 


I 


5116 


874 


17 


2354 


-46 


1404 


27 


484 


9 


7 


3619 


1307 


36 


1255 


35 


821 


23 


236 


7 


8 


1460 


608 


42 


385 


26 


263 


18 


204 


14 


. 9 


745 


380 


51 


122 


16 


185 


25 


58 


8 


10 


535 






223 


42 


117- 


22 


195 


36 


11 


226 






22 


10 


178 


79 


26 


12 


12 


22 






1 




3 


14 


19 


86 



Missouri 







> 1 


.50 


1.50- 


■1.01 


1.00 


-.71 


< .71 i 


Grade 


N 


n 


% 


n 


% 


n 


% 


n 


% 


1 


560 


87 


16 


118 


21 


106 


19 


251 


45 


2 


4035 


714 


18 


942 


23 


867 


21 


1513 


37 ■ 


3 


5043 


1659 


33 


1030 


20 


885 


18 


1469 


29 


4 


5497 


1003 


18 


1244 


23 


1027 


19 


2223 


40 


5 


5445 


1177 


22 


1192 


22 


932 


17 


2145 


39 


6 


4540 


1105 


24 


• 945 


21 


686 


15 


1804 


40 


7 


3468 


1038 


30 


599 


17 


438 


13 


1393 


40 


8 


2753 


786 


29 


504 


18 


' 371 


13 


1091 


40 


9 


721 


213 


30 


131 


18 


93 


13 


282 


39 


10 


352 


127 


36 


70 


20 


42 


12 


113 


32 


11 


102 


38 


37 


24 


23 


12 


12 


29 


28 
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TABLE 43 (conti) 



Nebraska 







> 1. 


50 


1.50 


-1.01 


1.00-.71 


< .71 


Grade 


N 


.n 


% 


n 


7. 


n 


% 


n 


% 


1 

X 


J / X 


ox 


Q 
O 


TX 
/J 


on 


67 


18 


200 


54 


I 


1904 


407 


21 


448 


24 


374 


20 


675 


35 


3 


2147 


440 


20 


482 


22 


360 


17 


865 


40 


4 


2253 


527 


23 


440 


20 


349 


15 


937 


42 


b 


2327 


663 


28 


463 


20 


311 


13 


890 


38 


6 


1973 


566 


29 


396 


20 


264 


13 


747 


38 


7 


1380 


488 


35 


223 


16 


153 


11 


516 


37 


8 


867 


257 


30 


133 


15 


91 


10 


386 


45 




393 


88 


22 


57 


15 


33 


8 


215 


55 


'.0 


172 


33 


19 


25 • 


15 


26 


15 


88 


51 


1 


142 


31 


22 


13 


9 


15 


11 


83 


58 


12 


■ 72 1 


12 


17 


8 


11 


12 


17 


40 


56 ■ 



North Dakota 







> 1 


.5 


1.5 


-1.1 


1.0-.6' 


< 


.6 


T dde 


N 

4 


n 


% 


n 


% 


n 


% 


n 


% 




930 


208 


22 


259 


28 


256 


28 


207 


22 




. 1041 


215 


21 


264 


25 


320 


31 


242 


23 




, 1091 . 


235 


22 


258 


24 


299 


27 


299 


27 




i 956 


190 


33 


193 


27 


262 


20 


311 


20 


h 


829 


190 


. 23 


171 


21 


217 


36 


251 


30 




534 


123 


28 


166 


15 


111 


26 


134 


31 


H 


' 387 


131 


34 


60 


16 


94 


24 


102 


26 


J 


97 


21 


22 


28 


29 


29 


30 


19 


• 20 


I 


65 


23 


35 


6 


9 


10 


15 


26 


- 40 


1 


2 














2 


100 
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TABLE 43 (cont.) 



Total Sample 







> 


1.50^ 


1.50- 


-1.01 


1.00- 


-.71 


< .71 


Grade 


N 

4 


n 


7. 


n 


•/. 


n 


% 


n 


% 


1 


12262 


^ 3102 


~\ 

25 


2927 


24 


3698 


30 


2537 


21 


2 


33878 


5957 


I 18 


7844 


23 


11893 


35 


8185 


24 


3 


33178 


7063 


1 


7080 


21 


11112 


33 


7923 


24 . 


4 


28081 


5480 


20 


6296 


22 


7985 


28 


8320 


30 


5 


27300 


• 6373 


1 23 


7506 


11 


6084 


22 


7338 


27 


6 


22488 


5563 


i 25 


5973 


11 


4421 


20 


6531 


29 


7 


13895 


4399 


' 32 


3196 


23 


2522 


18 


3778 


27 


8 


8731 


2900 


33 


1725 


20 


1441 


17 


2664- 


31 


9 


S066 


1947 


38 


853 


17 


826 


16 


1440 ■ 


28 


10 


2118 


639 


30 


490 


23 


349 


16 


640 


30 


11 


877 


227 


26 


112 


13 


285 


32 


253 


29 


12 


314 


102 


32 


43 


14 


54 


17 


115 


37 



'Sample includes Alabama, Arizona, Florida, Missouri, Nebraska and 
North Dakota. 

'The sample states did not use the same ranges of grade-equivalent 
scores but the ranges overlap sufficiently to permit aggregation. 
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TABLE 44 



Percent of Children Having Average Monthly Gains in 
Reading of 1.01 or More and l.Q'O or Less 
Derived from the Total Sample Data for 
the Years 1972,' 1973, and 1974 



Grade 




Samole 1972 


Total 


Sample 1973 


Total Sampl 


e 1974 


>^1.01 


£1.00 


>^1.01 


<_1.00 


>1.01 


<1.00 


1 


19 


81 


13 


87 


49 


51 


2 


34 


66 


31 


69 


41 


59 


3 


42 


58 


32 


68 


43 


57 


4 


38 


62 


32 


68 


42 


58 




30 


70 


30 


70 


50 

» 


50 


6 


36 


64 


30 


70 


52 


48 


7 


36 


64 


32 


68 


55 


45 


8 


32 


68 


37 


63 


53 


47 


9 


67 


33 


43 


57 


55 


45 


10 


54 


46 


44 


56 


53 


47 


11 


56 


44 


47 


53 


39 


61 


12 


64 


36 


59 


41 


46 


54 
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gain score distribution, etc. Regardless of the cause, however;, the dif- 
ference serves to highlight the futility of attempting to draw any in- 
ferences about the effects of Title I in general from data currently 
available in State Annual Evaluation Reports. 

As mentioried earlier, the data in Tables 37 through 44 present the 
only aggregations of adequately representative and valid achievement benefit 
data which could be made. Other reports, however, contained data which 
were judged to be adequate although presented in a format not compatible 
with that used by other states the same year. In order to give as complete 
a picture of Title I impact as possible, these data are presented, unaggregated, 
in Tables 45 through-^ 48. 
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TABLE 45 



1971 Reading Achievement Statistics: States with 
Adequate Data but Unique Reporting Formats 



California 1971 

Reading Achievement by Percent of Students in Each Range 
of Average Monthly Gain 



Grade 


N 


>1.4 . 


1.4-1.0 


.9-. 7 


<.7 


1 


• 13000 


5.6 


18.9 


23.7 


51.7 


2 


17281 


4.7 


29.1 


31.8 


34.4 


3 


27136 


2.9 


15.4 


41.8 


46.2 


4 


23426 


10.3 


20.8 


28.4 


37.8 


5 


21114 


7.6 


14.2 


36.9 


41.3 


6 


19488 


7.7 


14.4 


41.8 


36.1 


7 


2605 


15.2 


32.7 


34.9 


17.3 


8 


2210 


16.2 


21.6 


34.1 


29.7 


9 


3578 


20.6 


27.9 


33.5 


17.9 


10 


1638 


20.7 


33.4 


27.6 


18.3 


11 


586 


15.3 


24.9 


45.4 


14.3 


12 


278 


38.5 


41.7 


10.1 


9.7 



New Mexico 1971 

Reading Achievement by Percent of Districts in Each Range 
of Average Monthly Gain 



Number of 

Districts >1.4 1.4-1.1 . 1.0-.6 <.6 

71 8,5 35.2 39.4 16.9 
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TABLE 46 (cont.) 

New Mexico 1972 

Reading Achievement by Percent of Districts in Each Range 
of Average Monthly Gain 



>1.4 1.4-1.1 1.0-.6 <,6 

10 24.3 52.3 13.4 

New York 1972 

Amount of Gain and Average Monthly Gain for Students 
as a Function of the Number of Months Between 
Pre- and Posttest (Grades 1-6) 



Months Between 



Tests 


N 


Gain 


AMG 


.1 


4848 


3.60 


3.60 


2 


346 


6.00 


3.00 


3 


1754 


4 . 34 


1.45 


4 


754 


'4.26 


1.06 


5 


668 


6.64 


1.33 


.6 


2248 


6.94 


1.16. 


7 


6009 


9.41 


1.34 


8 


6538 


9.13 


1.14 


9 


10969 


7.65 


.85 


10 


11077 


9.14 


.91 


11 


38 


13.00 


1.18 


12 


25 


9.58 


.80 



Wisconsin 1972 

Reading Achievement by Percent of Students in Each Range 
of Average Monthly Gain 



Grade 

Level 0-.4 .5-. 7 .8-. 9 1-1.5 1.6-2 2,1-2.9 3 or More 



1 


25.96 


2 


8.81 


3 


5.63 


4 


12.41 


5 


1.50 


6 


3.68 


7 


16.39 


8 


1.37 


9 


15.15 


10 


0 


11 


33.33 


12 


0 



2.74 20.37 

8.15 12.86 

26.19 5.84 

11.04 21.34 

12.17 15.92 

19.40 9.70 
22.95 0 

15.07 16.00 
0 0 
0 0 
0 0 
33.33 0 



27.82 


15.66 


55.20 


6.69 


41.85 


13.49 


38.83 


14.27 


54.49 


9.93 


47.49 


6.35 


22.95 


10.38 


36.99 


20.55 


37.88 


9.09 


0 


63.64 


0 


0 


8.33 ' 


0 



6.13 1.31 

8.28 0 

7.00 0 

1.12 .99 

5.62 .37 

12.37 1.00 

0 27.32 

0 9.59 

19.70 18.18 

27.27 9.09 

40.00 26.67 

0 58.33 
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TABLE 47 

1973 Reading Achievement Statistics: States with Adequate 
Data but Unique Reporting Formats 



Arkansas 1973 

Number of Students with AMG's in Reading 
in Specified Ranges 

>.9 >9->l <*1 

2696 1718 1871 



Hawaii 1973 

Percent of Title I Students Who Achieved More Than 
Six-month Growth in Reading Achievement 
Reported by Districts 



Number Achieving % of Sample 

District More than 6 Months Achieving Goal 

Honolulu 577 50 

Central 144 73 

Leeward 303 47 

Windward ^ 201 58 

Hawaii^ 227 50 

Maui 136 63 

Kauai 79 80 

1. 5th .grade Pope and Waimanalo not included 

2. Excluded elementary schools which only had 3-to-4 
months pre-ppsttest span 



Kentucky 1973 

Reading Achievement as a Function of Gain 
and Average Monthly Gain 



Time Between 

N Pre and Post Gain AMG 

61,047 .77 1.05 1.36 
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TABLE 47 (cont.) 
Louisiana 1973 



1 7 



Reading Achievement as a Function of Changes 
in Percentile Rankings by Grades 



PERCENTILE BAND 1-5 



Grade 


N 


Pre 

No. 


%. 


Post 

No 


% 


Out 

No. 


% 


1 


1,662 


440 


26.5 


139 


8.4 


301 


18.1 


2 


7,969 


2,095 


26.3 


1,049 


13.2 


1,046 


13.1 


3 


9,190 


2,669 


29.0 


1,649 


17.9 


1,020 


11.1 


"4 


9", 180 


2,973 


32.4 


1,538 


16.8 


1,435 


15. 6 


5 


8,854 


3,025 


34.2 


2,236 


25,3 


789 


8.9 


6 


8,322 


2,674 


32.1 


2,242 


26.9 


432 


5.2 


7 


7,080 


2,487 


35.1 


1,857 


26.^2 


630 


8.9 


8 


5,685 


2,280 


40.1 


1,836 


32.3 


444 


7.8 


9 


4,155 


1,490 


35.9 


1,192 


28.7 


298 


7.2 


10 


2,5.09 


943 


37.6 


722 


28.8 


221 


8.8 


11 


1,424 


607 


42.6 


519 


36.4 


88 


6.2 


12 


791 


333 


42.1 


265 . 


33.5 


68 


8.6 






PERCENTILE 


BAND 1 - 


25 






1 


1, 662 


1,460 


87.8 


791 


47.6 


669 


40.2 


2 


7,969 


6,393 


80.2 


4,258 


53.4 


2,135 


26.8 


3 


9,190 


7,851 


85.4 


5,996 


65.2 


1,855 


20.2 


4 


9,180 


8,147 


88.7 , 


6,648 ^, 


72.4 


1,499 


16.3 


5 


. 8,854 


7,923 


89.5. 


7,011 " 


79.2 


912 


10.3 


6 


8,322 


7,458 


89.6 


6,500 


78.1 


958 


11.5 


7 


7,080 


6,461 


91.2 


5,461 


77.1 


1,000 


14.1 


8 


5,685 


5,143 


90.5 


4,597 


80.9 


546 


9.6 


9 


4,155 


3,786 


91.1 


3,315 


79.8 


471 


11.3 


10 


2,509 


2,113 


84.2 


1,958 


78.0 


155 


6.2 


11 


1,424 


1,290 


90.6 


1,166 


81.9 


124 


8.7 


12 


791 


693 


88.2 ■ 


656 


82.9 


42 


5.3 






PERCENTILE 


BAND 56 - 


99 







1 1,662 50 3.0 - 296 17.8 246 14.8+ 

2 7,969 380 4.8 999 12.5 619 7 7+ 

3 9,190 239 2.6 687 7.5 448 4*9 

4 9,180 101 1.1 387 4.2 286 3.'l • 

5 8,854 71 0.8 294 3.3" 223 2 5 

6 8,322 137 1.6 306 3.7 169 2.*0+ 

7 7,080 46 0.6 313 4.4 267 3.8 

8 5,685 47 0.-8--, 164 ^ 2.9 117 2*1 

9 4,155 25 0.6 \ 98 2.4 73 1*8 

10 2,509 32 1.3 I 94 3.7+ 62 2.'5 

11 1,^24 8 0.6 28 2.0 
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20 1.4 



^91 10 1.3 18 2.3 . 8 1.0 
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TABLE 47- 



1973 Reading Achievement Statistics: States with Adequate 
Data but Unique Reporting Formats 



Arkansas 1973 



f " Number of Students with AMG's in Reading 

in Specified Ranges 

>.9 ,9-,l <,1 

2696 1718 1871 

Hawaii 1973 

Percent of Title I Students Who Achieved More Than 
Six-month Growth in Reading Achievement 
Reported by Districts 



Number Achieving % of Sample 

District More than 6 Months - Achieving Goal 

Honolulu 577 ' 50 

Central 144 73 

Leeward 303 47 
Windward ^ .201 '58 

Hawaii^ 227 50 

Maui 136 63 

Kauai ' ' 79 80 

1. 5th grade Pope and Wairaanalo not included 

2. .Excluded elementary schools which only had 3-to-4 

months pre-posttest span 



Kentucky 1973 

Reading Achievement as a Function of Gain 
and Average Monthly Gain 



Time Between 

N Pre and Post Gain AMG 

61,047 .77 . 1.05 1.36 



116 

138 



TABLE 47 (cont.) 
Louisiana 1973 



Reading Achievement as a Function of Changes 
in Percentile Rankings by Grades 



^PERCENTILE BAND 1-5 



Pre Post Out 



Grade 


N 


No. 


% 


No 


% 


No. 


% • 


1 


1,662 


440 


26.5 


139 


8.4 


301 


18.1 


2 


7,969 


2,095 


26.3 


1,049 


13.2 


i,046 


13.1 


3 


9,190 


2,669 


29.0 


1 fi4Q 


1 7 Q 


1 090 


11 1 

XX . X 


4 


9,180 


2,973 


32.4 


1,538 


16,8 


1,435 


15.6 


5 


8,854 


3,025 


34.2 


2,236 


25 3 


IRQ 


A Q 
O . 7 


6 


8,322 


2,674 


32.1 


2,242 


26.9 






7 


7,080 


2,487 


35.1 


1^857 


26 . 2 




8 Q 

O . 7 


8 


5,685 


2,280 


40.1 


1,836 


32 .3 




7 8 


9 


4,155 


1,490 


35.9 


1,192 


28*7 


298 

^ y Kj 


7 9 


10 


2,509 


■ 943 


37.6 


722 


28.8 


221 


8 8 
o . o 


11 


1,424 


607 


42.6 


519 


36.4 


88 

KJ KJ 


fi 9 


12 


791 


333 


42.1 


265 






8 f\ 
o . p 






PERCENTILE 


BANT) 1 - 








1 


1,662 


■1,460 


87.8 


791 


47.6 


. 669 


40.2 


• 2 


7,969 


6,393 


80.2 


4,258 


53.4 


2,135 


26.8 


3 


9,190 


7,851 


85.4 


5,996 


65.2 


1,855 


20.2 


4 


9,180 


8,147 


88.7 


- 6,648 


72.4 


1,499 


16.3 


5 


8,854 


7,923 


89.5 


7,011 


79.2 


912 


10.3 


6 


8,322 


7,458 


89.6 


6,500 


78.1 


958 


- 11.5 


7 


7,080 


6,461 


91.2 


5,461 


77.1 


1,000 


14.1 


8 


5,685 


5,143 


90.5 


4,597 


80.9, 


546 


9.6 


9 


4,155 


3,786 


91.1 


3,315 


79.8 


471 


11.3 


10 


2,509 


2,113 


84.2 


1,958 


78.0 


155 


6.2 


11 


1,424 


1,290 


90.6 


1,166 


81.9 . 


124 


8.7 


12 


791 


693 


88.2 


656 


82.9 


42 


5.3- 






PERCENTILE 


BAND 56 - 


99 






1 


• 1,662 


50 


3.0 


296 


17.8 


246 


14.8+ 


2 


7,969 


380 


4.8 


999 


12.5 


619 


7:7+ 


3 


9,190 


239 


2.6 


687 


7.5 


448 


4.9 




9,180 


101 


1.1 


387 


4.2 


286 


3.1 


5 


8,854 


71 


0.8 


294 


3.3 


223 


2.5 


6 


8,322 


137 


1.6 


306 


3.7 ^ 


169 


2.0+ 


7 


7,080 


46 


0.6 


313 


4.4 


267 


3.8 


8 


5,685 


47 


0.8 


164 


2.9 


117 


2.1 


9 


4,1-55 


25 


0.6 


98 


2.4 


73 


1.8 


10 


2,509 


32 


1.3 


94 


3.7+ 


62 


2.5 


11 


1,424 


8 


0.6 


28 


2.0 


20 


1.4 




791 


10 


1.3 


18 


2.3 


8 


1.0 



ERIC 
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TABLE 47 (cont.) 

Ne\^ Mexico 1973 

Reading Achievement by Percent of Districts 
in Each Range of Average Monthly 
Gain 

>1.4 1.4-1>1 1,0-. 6 <.6 

10 22 36 32 
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TABLE 48 (cont.) 
Indiana 1974 



Reading Achievement- Pretj^st, Posttest, and- Gain 
Scores Expressed in Terms of Standard Scores 
and Percentiles 



Grade 



/ 



N 



Pre 
% S.S. 



Post 
% - S.S, 



Gain 
% S.S, 



4 


1394 


1 

i 

10 1 


53 


18 


58 


8 


5 


5 


1267 


20 ! 


68 


23. 


69 


3 


1 


6 


1283 


16 ' 


71 


20 


73 


4 


2 



Maine 1974 



Reading Achievement Pretest, Posttest and Gain 
Scores Expressed in Terms of 
Standard Scores 



Grade 


N 


Pre 


Post 


Gain 


2 


47 


137.83 


144.74 


6.91 


3 


113 


146.21 


152.13 


5.92 


5 


127 


425.40 


430.07 


4.67 


6 


162 


428.93 


433.75 


''4.82 



Ohio 1974 

Number and Percent of Students Scoring Within 
Specified Average Monthly Gain Ranges in Reading 



Grade 
Band 


N 


1 

n 


.5 

% 


1.1- 

n 


-1.4 

% 


.6-1.0 

n /o 


.5 

n 


% 


PreK-K 


2381 


740 


31 


575 


24 


635 


I 

27 


431 


18 


1-3 


52821 


19179 


36 


10768 


20 


■ 

12611 


24 


10263 


20 


4-6 


25943 


. 11074 


43 


3830 


15 


5078 


20 


5761 


22 


7-9 


2264 


862 


38 


345 


. 15 


453 


20 


604 


27 
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TABLE 48 (cont.) 



South Dakota 1974 

Reading Achievement as a Function of Gain in 
Grade-Equivalent Scores 



Grade Band 


N 


Gain 


1-3 


1452 


1.03 


4-6 


2424 


1.00 


7-8 


792 


1.00 



■ Virginia 1974 

Reading Achievement as a Function of Average 
Monthly Gain Reported by Type of 
Standardized Achievement Test 





AMG, 


AMG 


Grade ^ 


Test l2 


Test 2 


2 


l."2 


.9 


3 


1.3 


.9 


4 


1.2 


1.2 


5 


1.2 


1.0 


6 


1.2 


1.2 


7 


1.3 


1.3 


8 


1.0 


.9 



1. Total N = 27,957 

2. Tests 1 and. 2 are the CAT and MAT. It was not 
clear from the report which column represented 
data from which test: 
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Needs Assessment and Parent Advisory Councils 

The following chapter deals with two more important issues that 
were addressed in the State Reports: needs assessment and Parent Advisory 
Councils. 

Needs Assessment 

An average of 32% of the 1971, 1972, 1973, and 1974 State Reports 
mentioned needs assessment. By referring to Table 49, it can be seen that 
the needs assessment pattern is similar for each of the four years. The 
smaller number of states reporting on needs assessment in 1974 reflects 
the fact that only 30 State Reports were available at the time of aggrega- 
tion. 

Information in the State Reports related to the needs assessment 
process was grouped under three main topics: the people involved, the 
instruments used, and the results of the process. Within each of these 
general categories, the number of states th'at listed specific people, in- 
struments, or areas of need was then tabulated. 

The State Reports indicated that teachers, counselors, psychologists, 
social workers, parents, and Parent Advisory Councils were involved in 
ascertaining needs with teachers clearly playing the most important role. 
A variety of instruments were mentioned in the reports as tools for 
evaluating areas of need. The instruments included most prominently 
standardized tests, with teacher-made tests, grades, measures of classroom 
performance, cumulative records, attitude inventories, health and 
attendance records, and home visit data also used although with signifi-^ 
cantly less frequency. The areas of need indentified in the State Reports 
included reading, language arts, math, social studies, and science 
as well as nonacademic areas such as social and emotional development 
and health. The basic skill areas of reading and math* were the most 
frequently reported upon - as might be expected from the relatively common 
use of standardized tests for needs determination. 

Parent Advisory Councils 

Table 50 presents a breakdom of the number of states reporting 
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TABLE 49 



The Number of States Reporting 
Information Related to Needs Assessment 
Presented by Fiscal Year ' 





1971 


1972 


1973 


1974 


People involved in needs assessment 










Teachers 


8 


9 


10 


3 


, Counselors 


2 


5 


5 


2 


Psychologists 


1 


4 


3 


0 ■ 


Social Workers 


0 


3 


2 


1 


Parents 


2 


6 


6 


2 


Advisory Councils 


3 


5 


3 


2 


Instruments used to assess needs: 










Q 1" an T*rl n t l"o o l~ c? 


1 A 


1 A 

±H 


11 


0 


Teacher-made tests 


6 


4 


5 


, 2 


Grades 


4 


2 


3 


1 


Classroom Performance 


6 


7 


3 


2 


Cumulative Record 


2 


4 


4 


0 


Attitude Inventories 


5 


1 


3 


0 


Health Records 


6 


2 


1 


1 


Attendance Records 


6 


2 


3 


0 


Home Visit Data 


0 


1 


0 


0 


Areas of need ^determined by assessment 










process: 










Reading 


11 


9 


6 


5 


Language Arts 


8 


5 


6 


4 


Math 


10 


6 


6 


4 


Social Studies 


2 


2 


1 


1 


Science 


1 


2 


1 


1 


Social and Emotional Need 


7 


4 


2 


3 


Health 


6 


4 


3 


3 



Note , In 1971, 17 of 53 state reports included informatio.n 
regarding needs assessment. Sixteen of 52 reports in 1972, 
15 of 42 reports in 1973, and 8 of 30 reports in 1974 had 
need s ass es smen t inf ormat ion . 
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TABLE 50 

The Number of States Reporting Information Related to 
Parent Advisory Councils Presented by Fiscal Year 





1971 


1972 


1973 


1974 


Number of PACs 


10 


O 


c 

J 


1 


Number of Parents 


3 


8 


14 


7 


Number of People on 
PACs 


2 


4 


5 


4 


Number of Meetings 
for each PAC 


4 


11 


13 


8 


Membership. Breakdown 


8 




8 


4 


Title I Parents 


8 




8 


5 


School Personnel 


6 




8 


3 


Others 


6 




7 


4 


How PAC Membership was 
Selected 


2 


2 


3 


4 


Activities 


9 


12 


15 


8 


Determine Needs 


3 


7 


8 


6 


Plan Projects 


8 


12 


15 


8 


Visit Projects 


2 


3 


4 


5 


Volunteers as Aides 


3 


3 


2 


3 


Evaluation 


5 


11 


11 


7 


Disseminate Infor- 
mation 


6 


5 


4 


7 


Funding PAC's 


1 


2' 


3 


2 



Note. In 1971, 19 of 53 state reports included information regarding 
Parent Advisory Councils. Nineteen of 52 reports in 1972, 22 of 42 
reports in 1973, and 15 of 30 reports in 1974 had information on PACs. 
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various types of information pertaining to Parent Advisory Councils. Some 
State Reports described parental involvement in general, but did not refer 
to PACs specifically. Other reports mentioned the fact that PACs existed 
in their state, but provided no further relevant information. The State 
Reports containing these brief references to parents or PACs were not in- 
eluded in the data tabulation. 

Table 50 shows how many states reported the number of PACs operating 
in their state, the numbers of meetings held by the PACs, the number of 
parents on the councils, and the number of people in general on the councils. 
The number of states that indicated the composition of their PACs is also 
tabulated. To be included in this count a state need not have given a 
numerical breakdown of membership, but need only have listed the various 
types of people serving on the councils. The states that described how 
their PAC members were selected are also tallied. 

The number of states that identified the activities engaged in 
by the PACs is also indicated in Table 50 with a breakdown by type of 
activity. The way in which the PACs were funded was mentioned by some 
states and their number is reported. 

There appears to be some trend toward increased reporting of PAC- 
related information and toward providing more detailed descriptions of 
PACs although the majority of the State Reports made no mention of this 
important ingredient of Title I projects^* 
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Conclusions and Recoiigaandations 

Each of the preceding sections of this chapter presented some 
conclusions and recommendations derived from analyses of past' reports. 
All oj^ this Information Is summarized here as clearly and concisely as 
possible, 

Participation 

In order to form an accurate picture of the extent and nature of 
Title I participation, uniform and reliable data are required. It Is 
necessary to know at least how many children are being served, and whether 
they are being served In the regular term, summer term, or yearlong. For 
these categories of participation, undupllcated counts are required. Most 
states are currently providing some of- this Information In their State 
Reports, The breakdowns they use, however, and the categories on which 
they report are sufficiently Inconsistent from state to state so that It 
Is not possible to derive a nationally representative picture of Title I 
participation. 

Information concerning the number of participants being served In 
certain Instructional and supportive programs Is also desirable. Since 
the legislation suggests that Title I funds be spent primarily In the 
basic skill areas. It Is Important to determine whether a large proportion 
of Title I participants are being served by these programs. This Infor- 
mation should be provided by a duplicated count. In which participants 
are counted once for each program In which they participate. The pro- 
gram participation data should be presented by standard program categories 
so that aggregation over all states Is possible,. Several State Reports 
present program data In terms of standard categories but the practice has 
by no means been universally adopted. 

If basic Information can be provided in all State Title I Evaluation 
Reports, it will be possible to form a national picture of Title I partici- 
pation. In Section 512 of the Education Amendments of 1974 (P,L, 93-380), 
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Congress mandated a study to recommend the format for a consolidated 
state report on participants arid expenditures in all federally funded 
programs. The results of that study will, of course, influence which 
types of participation and expenditure data are requested in State Title I 
Annual Evaluation Reports. 

Resource Allocation 

Unquestionably, the greatest difficulty encountered in- assembling 
a national picture of Title I resource allocation from State Title I 
Annual Evaluation Reports is that many reports contain no data at all 
on allocations or expenditures. Resource allocation is an important 
policy-relevant issue, so it seems appropriate to recommend that these 
State Reports provide at least minimal information on the expenditure 
of Title I funds. 

Since confusion now exists in the reports between expenditures 
and allocations, and since it is not always clear what funds have been 
included in or excluded from the reported amounts, complete labeling is 
particularly important. It would be most useful for reports to show total 
state allocations and total expenditures, in each case including ad- 
ministrative funds, carr:y-over funds, and funds under Parts B and C of 
Title I. If this information were reported consistently, comparable 
fl ^ures could be extracted from the reports for aggregation across states. 
Again, results -of the mandated study concerning the format for reporting 
participation and funds relevant to federall> funded programs will affect 
the future content of State Reports. 

Several State Reports currently present expenditures in break- 
downs by type of service for summer and regular sessions, and by standard 
accounting categories. These data are particularly useful in identifying 
program emphasis and major expenses. A breakdown of per-pupil expendi- 
tures by standard program area is also very useful since these figures 
make it possible to judge how expensive it is to offer various Title I 
programs or services. However, too few states report expenditures by 
program areas or by accounting categories to suggest that it would be 
feasible for most states to produce this type of information. Although 
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per-pupil expenditures by program also are infrequently reported now, 
these figures ate easily produced if information is available on both 
expenditures and participation by program area. Assuming that these data 
could be produced by the states, a reliable national picture could be 
formed of how Title I funds are spent by the states. 

Impact 

The current State Reports do not achieve their potential as. sources 
of program evaluation data. A small number of states reported data 
that were adequately reliable and valid. Within this group, an even 
smaller ntimber of states shared a common format. Of the 17 states that 
did have representative data in FY 71, 72, 73, and 74 but could not be 
included in the across-state aggregations, seven did not include any 
grade-level breakdowns and three included grade band breakdowns. For 
these states, the difficulty in interpreting. achievement data pooled 
across grade levels was compounded by the use of grad^-equivalent scores 
which precluded any assessment of the pre- or posttest status of program 
participants. For data in future reports to be more useful, it is 
necessary that they be presented by grade level. 

During the policy interviews with federal-level officials con- 

i 

ducted in Phase I, many of those interviewed- expressed the desire to 
see impact data reported at the project levei. To do so would require 
that data be presented for each unit of one or more instructional or 
service areas under a single administration . Officials hoped that by having 
evaluation information at the project level, successful and unsuccessful 
projects could be identified. Successful ones would then be disseminated 
and unsuccessful ones which had received a fair test would be changed 
so that the Title I program would be continually improved. , Especially at 
the state level, the desirability of project-level impact reporting seems 
clear, with appropriate summary tabulations being forwarded to the federal 
level. 

Many of the present evaluation efforts demonstrate an inadequate 
awareness concerning the methodology and design required to collect 
meaningful impact data. If norm-referenced evaluation models continue 
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to be used by the majority of states, evaluators must be made cognizant , 
of the restrictions inherent in these designs. For normative data to 
provide valid baseline indications, testing can only be'conducted at 
times corresponding to those at which the actual normative data were 
collected. Further, if a fall pretest and spring post test are desired^ 
(as is most often the case) then the choice of tests is restricted "^since 
only a few instruments have appropriate normative data points. Ciosely 
related to these issues afe the difficulties in measuring program impact 
using grade-equivalent scores which arise from scaling irregularities 
and curve-fitting interpolation procedures (see Tallmadge & Horst, 1974.) 

Needs Assessment ^ 

The area of needs assessment .was not addressed thoroughly, if at 
all, in most of the State Reports. Policy makers have indicated that the 
three primary factors to be included in a description of needs assessment 
are: the people involved, the instruments and procedutas used, and the 
results. • State Reports 'which contained some information on needs 
assessment seldom included discussions^ of all three of these factors. 

Data concerning the outcome of the needs assessment process can 
be compared with figures on resource allocation and program participation 
in order to judge whether the children being served are those with the 
greatest needs and whether their identified needs are those being served 
by the programs in which they participate. 

Parent Advisory Councils 

Title I legislation is particularly, clear in regard to the number, 
composition, and functions of Parent Advisory . Councils (PACs) . Each 
LEA must establish an advisory council for the entire school distric't 
and each school served by Title I must have a PAC. The majority of, the 
Council members must be parents of children served by^Title I. Each 
PAC is to advise in the planning, implementation, and evaluation of pro- 
grams and pro jec^sT^"^"-^.,^^^^^ ^ 

Since these requirement's- are clearly spelled out, it seems important 
to know whether there are the legally required number of PACs, whether 
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the required number of parents of Title I children are represented on the 
Councils, and in what activities the PACs have participated. The minimal 
information to be requested from the SEAs would be a report of (a) the 
percentage of schools that have PACs, (b) the percentage of PACs that 
are legally constituted, (c) the numbers and types of people serving the 
Councils, and_(d) the activities engaged in by the Councils. 
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IV. PHASE III 



Summary of Phase III Tasks 

The first task of Phase III involved interviewing the State Title I 
Coordinators, by telephone in order to determine the feasibility of im- 
plementing recommended evaluation and reporting practices in their states. 
During these discussions, interviewers also attempted to find out how much 
authority or influence the SEA exercised over its LEAs, and to clarify 
the reasons behind state practices which differed significantly from the 
recommended practices. 

The State Coordinators were notified in advance of the telephone calls 
by a letter (see Appendix E) which described the purposes of the telephone 
calLs^ and_ the topics which the interviews would cover. A brief summaryr 
of the project and copies of the critical summaries of that state's Annual 
Reports were sent with the letter. 

The telephone interviews were conducted by two-person teams with one 
member interviewing the Coordinator while the other member took notes- 
Questions were asked in the areas of participation, expenditures, impact, 
needs assessment, and Parent Advisory Councils- However, no two states 
were asked the same questions, since the topics covered in the interviews 
were determined both by practices already existing in the state, as evi- 
denced in the State Report, and by the Coordinator's responses to initial 
questions about Title I participation- Because the content of the inter- 
views varied widely, no attempt was made to quantify the states' responses- 
After all states had responded to the interview questions, three 
preliminary evaluation and reporting models were developed. These models 
were identical with the exception of their impact assessment strategies, 
which were labeled Norm-referenced (Model A), Control Group (Model B) , and 
Special Regression> (Model C)- The latter two had variations to allow for 
the use of either standardized achievement tests or criterion- or objectives- 
referenced tests- The Control Group Model was appropriate for use with a 
posttest-only comparison, with or without matching, and with or without 
covariance adjustment procedures - 
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For each model, repotting forms and instructions were developed 
for project reports to LEAs, LEA reports to SEAs, and SEA reports to the 
U. S. Office of Education. Summary information about individual projects 
was reported at all levels. The thre& models were designed so that all 
data, including impact data, could be aggregated across as, well as within 
models. The reporting system thus provided project-level information about 
the children served by Title I projects, the types of services provided, 
the costs of the services, and their impact. In addition, the system pro- 
vided a state summary of information about Title I personnel, preservice 
and inservice training, and school and district Parent Advisory Councils. 
During Phase IV, nine SEAs examined the reporting system to determine the 
feasibility of implementing such a system in their states. 

Telephone Interviews 

The first major task of Phase III was the interviewing of State 
Title I Coordinators by telephone to explore the feasibility of implement- 
ing various evaluation and reporting practices in their states- To alert 
the Coordinators to the purposes of the study and the telephci.e interview, 
letters were sent to each of them approximately two weeks before the ini- 
tial attempt was made to contact them. A copy of the letter is included 
in Appendix E. 

The^-elephone calls were made between February 6th and March 5th 
to all fifty states and to the Bureau of Indian Affairs, the District of 
Columbia, Puerto Rico, and the Virgin Islands. Most frequently, the Co- 
ordinator participated in the telephone conversation, but often he was 
joined by the Title I Evaluator. Occasionally the Coordinator did not 
participate, designating the Evaluator as the person most appropriate 
to respond to the interview questions. 

Only one state refused to answer' questions by telephone and that 
state did send a written response to the nine questions contained in the 
pre-interview letter. While all other State Departments cooperated in 
the interviews, some were clearly hostile to the objectives of the study, 
to the federal intervention they felt the interviews represented, and even 
to evaluation in general. Several interviewees were quick to point out 
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that a number of federal evaluation efforts, which they felt were ill- 
conceived, had been tried out in the past and had failed. They did not 
want to be involved in similar endeavors. 

Because of the wide variety of responses received from the various 
states, it is not possible to describe any kind of general reaction toward 
the interviews or the study itself. In most instances where hostility was 
encountered, however, there appeared -to be a substantial increase in in- 
terest as it became clear to the state representatives that there would be 
extensive involvement of SEA and LEA personnel in • develop ing^'tlie new system, 
that it would not involve a massive data collect ioji Instrument such as that 
employed by ESS and CPIR, and that there wotild'^ be no immediate mandate to 
adopt a new and untried system. At least a few of the initially hostile 
state representatives expressed a positive attitude toward the study and 
participation in it by the time the interview was completed. 

In preparing for the interviews, three separate sources of infor- 
mation were considered: (a) past State Title I Annual Evaluation Reports, 
(b) suggestions made by the study's Advisory Panel (see Acknowledgments) 
with respect to future evaluation and reporting systems, and (c) experi- 
mental design and statistical considerations relevant to data reliability, 
validity, and potential for aggregation. The manner in which each of these 
sources was used is described in the following paragraphs. 

From past State Reports it was possible to determine the entire gamut 
of evaluation and reporting practices. Of particular interest for the 
telephone interviews were those practices which were most complete and 
easily interpretable. While no decision was made that the new reporting 
system would call for such complete infomation on all relevant topics. 
It seemed appropriate to assess the feasibility of collecting this type 
of data in all states. 

The Advisory Panel was particularly helpful in pointing out several 
specific problem areas which were subsequently investigated during 
the telephone interviews. Perhaps most useful were the suggestions per- 
taining to financial matters and the questions raised with respect to 
reporting on needs assessment and parent involvement issues. All of the 
panel's thoughts on these issues were reflected in the interviews. 
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The experimental design and statistical considerations Relevant to 
valid evaluations at the project level had been well thought through prior 
to the present study (see Horst, Tallmadge, & Wood, 1975). Problems 
associated with the aggregation of impact data, however, had barely been 
considered and required a substantial amount of effort to resolve. All of 
the design and statistical issues, whether related to the validity of 
inferences drawn at the project level or to state aggregation problems, 
had implications for the still-to-be-developed model evaluation and 
reporting systems- These implications often took the form of procedures 
which evaluators at the proiect, LEA, and SEA levels would have to adopt 
if the systems were to be effectively implemented. The feasibility of 
the procedures and the willingness of evaluators to adopt them thus 
became issues which had to be explored during the telephone interview. 

All of the interviews were conducted by two-person RMC teams with 
one member asking the questions while the other took notes. Team members 
were rotated, and each member of the staff spent approximately equal 
amounts of time in interviewing and note-taking roles. Initial attempts 
to talk to each Title I Coordinator were made exactly two weeks^ after the 
date of the letter which was sent. Only about half of the calls were 
successfully completed on the first attempt, however, and in a few instances 
it required as much as two weeks of almost daily telephoning before an 
interview could be completed. 

Questions were asked in the five areas of participation, expendi- 
tures, impact, needs assessment, and Parent Ad^visoiry Councils. No two 
states were asked the same questions, however. Rather, the procedure was 
generally to open up the discussion by explaining briefly the purpose of the 
study and asking an initial question- or two in the participation area. 
Any responses of a non-routine nature would then be pursued until the 
interviewer felt that he or she had a reasonably clear picture of the 
state's attitude tpward the issue and of what could and could not be 
accomplished in the way of a revised evaluation and reporting system- 
Then the interviewer would move on to the next topic - 
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Occasionally, .the interviewee had misinterpreted the purpose of the 
questions contained in the pre- interview letter and attempted to provide 
clear and concise answers to those questions. When it was pointed out that 
those questions had been included only as examples of the kinds of issues 
which would be discussed it became clear that some interviewees were not 
prepared for such open-ended discussions. In several instances the inter- 
viewer was requested to call back at a later date. 

Because the interviews were largely unstructured, it was neither 

meaningful nor feasible to tally the responses of the states according to 

their position on various issues. While, except in the cases of the most 

uncooperative states, the interviewer came away from each telephone call 

feeling that a good understanding of the state's position on each issue 

had been obtained, a substantial amount of judgment, interpretation, and 

"reading between the lines" was frequently required. For this reason the 

following discussions of the five content areas are largely subjective 

and are cast in terms of the conclusions reached from the interviews more 

than in terms of specific responses to specific questions, 
/ 

SEA responses to questions about reporting participation infoirmation 

indicated that it would be feasible to include in the model reporting 

\ 

system unduplicated counts of students in various grade levels for public 
and nonpublic schools, and to report duplicated counts of children who 
receive reading, mathematics, other instruction, and supportive services. 

Although many Coordinators indicated that they could and would be 
willing to report these data in a more detailed program breakdown (such 
as that used in the CPIR) , members of the Advisory Panel and several Co- 
ordinators indicated that a very specific breakdown would often be in- 
accurate. The more general breakdown into four categories appeared to 
be acceptable to almost all SEAs and to satisfy the needs of policy makers 
for participation data. 

These breakdowns (by grade level and by program area) indicate the 
areas and levels in which most Title I children are being served. The 
information is also important in determining the reliability of impact data 
presented for reading and mathematics programs. 
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Although the content of the telephone interviews cannot be rigor- 
ously categorized and analyzed, a study of all the responses indicates 
that the states will accept and use a model reporting system which is based 
upon the separate presentation of impact and expenditure data from each 
Title I project in a state. Since decision makers have stressed the impor- 
tance of receiving project level information, the models developed by, RMC 
require the reporting of expenditure and impact data separately for each 
Title I project as well as in a statewide aggregation. Discussions with 
the Advisory Panel and xd.th State Coordinators about the reporting of ex- 
penditure data resulted in a decision to include minimal expenditure data 
in the reporting models. There appeared^ to be widespread feeling that 
Title I Annual Evaluation Reports must be submitted too early to include 
complete and accurate financial information, and that policy makers could 
always obtain this information from the Annual Financial Reports. In 
addition, many Coordinators agreed that it was extremely difficult, if 
not impossible, to accurately report expenditures for each program area or 
separ-ately for nonpublic schools Based on these discussions, the decision 
was made to restrict expenditure data in the reporting models to a minimum 
of readily available information, i.e., total and per-pupil costs. at each 
level (project, LEA, and State). ' 

Discussions with the State Title I Coordinators about reporting im- 
pact data focused on the issue of standardizing project evaluation prac- 
tices, and Coordinators' responses to this issue were quite varied. It 
appears that many Coordinators would consider requiring or recommending 
standard evaluation procedures to their LEAs and Title I projects. How- 
ever, a substantial number of State Title I Coordinators were ^clearly 
opposed to the adoption of standard evaluation practices in their states. 
It is hoped that these Coordinators may be persuaded to view the prospect 
more favorably after further discussion. Since adherence to one of the 
models is essential for each project and LEA if data are to be valid, re- 
liable, and aggregatable across the nation, KMC will continue efforts 
to persuade SEA and LEA personnel of the benefits to be gained from the ^ 
use of standard evaluation practices. 

Other questions related to program impact were concerned with the 
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frequency of use of criterion- or objectives-referenced tests, arid the 
Coordinators' responses to these questions indicated that in only a few 
states are these tests used extensively. However, it was clearly necessary 
to develop reporting models for projects using criterion- or objectives- 
referenced tests, so that the results of these tests could be combined ' 
with data based upon the more widely used standardized achievement tests. 

Coordinators' responses to questions about needs assessment pro- 
cedures indicated that almost all of the State Title I programs concentrate 
upon eliminating reading and math deficiencies, but that the procedures 
for identifying these deficiencies vary widely across and within the states. 
For example, certain states use an elaborate sequence of activities to 
conduct their needs assessments, while other assessments consist of only 
one or two steps. Because of the variety of needs assessment practices and 
because Coordinators appeared to have little interest in adopting standard 
practices across their states, it appeared impossible to develop a standard 
format for reporting needs assessment procedures and results. Since the 
purpose of including this information in the evaluation reports is to de- 
termine whether or not programs are serving the children who need services, 
the decision was made to rely on pretest scores to provide this information 
rather than attempt to develop acceptable standard needs assessment pro- 
cedures. Our discussions about needs assessments did indicate that most 
assessments include the use of standardized tests, and that this informa- 
tion could be included in the reports. 

The last area of discussion with the Coordinators usually concerned 
the composition, selection procedures, and activities of Parent Advisory 
Councils. Although the State Coordinators indicated that they could in- 
clude thi" type of information in their reports, they seriously questioned 
its validity- Also, a few Coordinators questioned whether an evaluation 
report was the appropriate vehicle for documenting compliance with the 
legislation. 

Although it is not possible to measure objectively the results of 
these interviews, it appears that they accomplished two major goals which 
are closely related to the successful development of the model evaluation 
and reporting system. First, the State Title I Coordinators were informed 
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about the purpose of our project and many of them were persuaded of its 
potential .for improving evaluations' and reporting of Title I programs^ 
Rapport was established between the interviewers and many State Coordina- 
tors which should aid in the acceptance and applicatinn of the reporting 
models by the states. 

Second, useful information was obtained about the feasi- 
bility of including in the reporting models certain types of data on stu- 
dent participation, expenditures, program impact, needs assessments, and 
Parent Advisory Councils. Our conclusion, based upon this information, 
is that sufficient data can be obtained from all Title I projects to per- 
mit the identification of successful Title I projects. In addition, it 
appears that most states will support the application of the new evalua- 
tion and reporting models if an intensive effort is made to persuade Title I 
Coordinators of the models' validity and usefulness. 
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The Models 



Three evaluation and reporting models were developed, all of 
which are identical except for their intact assessment strategy. Tenta- 
tively named for these strategies, the models are: Model A, the Norm- 
referenced Model; Model B, the Control Group Model; and Model C, the 
Special Regression Model. The Control Group Model and the Special Regres- 
sion Model have variations depending on whether standardized achievement 
tests or criterion- or objectives-referenced tests are used. The Control 
Group Model has additional variations in that it may be used with posttest-., 
only comparisons, with or without matching, and with analysis of covariance 
procedures - 

For each model, reporting forms were developed for reports (a) 
from projects to LEAs, (b) from LEAs to SEAs, and (c) from SEAs to the 
U. S. Office of Education. In addition to the reporting forms, implemen- 
tation procedures were developed for each of the models. These procedures 
were designed to provide step-by-step guidance to evaluators at the project, 
LEA, and SEA levels and were written in such *a way as to minimize the 
amount of technical expertise required for their execution. Perhaps the 
most significant feature of these reporting forms was that summary infor- 
' mat ion about individual projects was reported at all levels. While this 
represented a radical departure from previous reporting practices, it 
fulfilled the expressed information needc of the educational policy makers 
interviewed during Phase I of this study and appeared feasible, based on 
the analyses undertaken during Phase III. 

Another significant feature of the overall evaluation and reporting 
system was that the three models were worked out so that data (including 
impact data) can be aggregated across, as well as within models, thereby 
enabling the compilation of meaningful statewide and nationwide summaries. 

Before discussing the details of each model, it is appropriate to 
describe exactly what the proposed evaluation and reporting system is in- 
tended to do and what it is not intended to do. Perhaps the most important 
distinction to be made here is that the system is outcome, rather than 
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process, oriented. It provides some project-level descriptive informa- 
tion relating to the children served by Title I projects, the types of 
services provided, and the costs associated with their delivery. It also 
encompasses infoinnatlon about Parent Advisory Councils and in-service 
training. It does not, however, attempt to deal with compliance with 
existing legislation, regulations, program criteria, or guidelines at 
the project or even the LEA level. These issues, it has been concluded, 
cannot be adequately assessed through self-reporting procedures. 

State-level data are, for the most part, generated through 
aggregating information from project and LEA reports, and only informa- 
tion which is useful at those levels is collected. Data requirements 
are kept to the absolute minimum which is consistent with the objective 
of obtaining a meaningful, interpretable picture of Title I impact. Every 
attempt, furthermoife, was made to avoid* duplication with other data collec- 
tion and reporting requirements. Thus, financial infoirmation was restricted 
to total-project and per-pupil costs, and information about participants 
was limited to their numbers, grade levels, and test scores. 

The data called for by the proposed system does more than provide 
evidence regarding overall effectiveness of the Title I program. The 
system permits analyses of project-level relationships among costs, achieve- 
ment gains, hours of intervention, g;rade. levels , instructor-pupil ratios, 
and initial degree of educational need. It will, then, enable investiga- 
tion of most of the major and minor concerns expressed by the educational 
policy makers interviewed during Phase I of the study. 

Model A, the Norm-referenced Model, is expected to be the most 
widely adoptable of the models since it closely resembles commonly em- 
ployed, current evaluation strategies. The evaluation design itself 
deviates from current practice in just two respects: (a) it makes use of 
standard scores and percentiles rather than grade-equivalent scores, and 
(b) it requires that testing be done on dates corresponding to real 
normative data points rather than whenever testing is convenient. One 
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additional requirement' of this model is that the pretest be administered 
to the project participants after they have been selected. Pupils cannot 
be selected fpr participation based on their pretest scores. 

The reporting part of Model A requires the use of standard formats 
at the project, LEA, and SEA levels to facilitate the aggregation of data 
at each higher level. The forms were kept quite simple to minimize 
the burden imposed on evaluation personnel at each level. 

Model B, the Control Group Model, is substantially more difficult 
to implement than Modfel A because of the difficulty in assembling a suit- 
able control group. As pointed out in Horst, Tallmadge, and Wood (1975), 
the assignment of children to treatment and control groups must be random 
or "random-in--ef f ect" (it 'must be possible to consider the two groups as 
random samples from a single population). It is not possible to construct 
suitable control groups by post hoc matching procedures although this is 
a very common error. 

If a suitable control group exists, the problems of implementing 
Model B are substantially reduced. Furthermore the Model has several 
advantages over Model A including, most notably, the greater degree of 
confidence which can be placed in inferences drawn from it, and the fact 
that it is adaptable to use with either standardized achievement tests or 
criterion- or objectives-referenced measures. While use of the latter 
type of instrument does complicate the data aggregation problem somewhat, 
where a decision has been made to use criterion- or objectives-referenced 
tests.. Model B is clearly to be preferred over Model C, the only other 
available alternative. 

The reporting forms for Model B are somewhat more complex than for 
Model A. The increased complexity, however, stems largely frota the in- 
corporation of provisions for analysis of covariance adjustments. Presum- 
ably, these adjustments will be uncommon so that the actual reporting 
requirements will be less complex than they appear. 

An additional complication appears on the forms for Model B2, the 
variation which employs criterion- or objectives-referenced tests. The 
same complication exists in Model C2 and relates to the problem of 
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expressing the results of all evaluation models in a common metric — 
namely, the national norms • The use of this metric presents no problem 
when standardized achievement tests are used, but does add complications 
when evaluations employ tests without normative data. This issue is 
addressed separately at the end of this section. 

Model G, the Special Regression Model, should be regarded as the 
least desirable of the three models primarily because of the potential it 
has for misinterpretation. While it is fundamentally sound statistically 
and in terms of experimental design considerations, it is not always clear 
whether a significant difference found with the model should be attributed 
to a treatment effect or to some other influence such as a naturally 
occur ing curvilinear regression of post- on pretest scores. 

As included in the reporting system, the Model incorporates two 
different evaluation designs, the Regression-discontinuity design and the 
Regression Projection design. If the results from the two analytic ap- 
proaches are in agreement, the attribution of a significant finding to a 
treatment effect appears to be clearly justified. The main problem seems 
to arise where the Regression Projection design produces a significant 
finding while the Regression-discontinuity design does not. While this 
pattern of relationships would be predicted under certain circumstances 
including, most notably, a project in which the amount of special help 
given to individual pupils was proportional to their educational need, 
even under these circumstances the plausibility of possible alternate 
explanations could not be ignored. 

Where this kind of ambiguity exists, judgments will have to be 
made as to whether or not observed effects should be attributed to project 
impact. This is a complex technical judgment for which some guidelines 
can be provided but for which it will not be possible to establish non- 
controversial decision criteria. Unfortunately, it is not clear that 
those individuals who will be called upon to make the judgments will have 
the technical expertise required to interpret the available evidence cor- 
rectly or a sufficient degree of scientific detachment to choose an 
alternative explanation if attributing observed effects to project in- 
fluences would place their district or state in a more favorable light. 
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Despite these deficiencies, Model C represents the only viable 
technique for conducting evaluations with criterion- or objectives-referenced 
tests when a control group is not available.- Details regarding the Model 
and associated implementation procedures are contained in Horst, Tallmadge, 
and Wood (1975). It should be pointed out here, however, that the design , 
serves as its own needs assessment procedure in that all pupils scoring 
below some selected Xevel on a pretest are assigned to the treatment group 
while pupils above that level are not assigned to the treatment and serve, 
in effect, as a comparison' group. 

As pointed out earlier, one major objective for the overall evalu- 
ation and reporting system was that of enabling the aggregation of data 
across schools within districts, across school districts within states, 
and across states within the nation. At the same time, thejre was a strong 
desire to provide as much flexibility as possible by allowing local agencies 
to make their own choices as to which evaluation models and which test 
instruments they felt were most appropriate for their particular contexts. 
During Phase III, then, efforts were made to develop strategies for enabling 
both of these objectives to be met. 

The problem of aggregating data across models existed only in the 
area of impact data since it was planned from the beginning that standard 
formats would be used for reporting participation, resource allocation, 
and other input and process information. Even in the impact area, the 
problem was not difficult except for the issue of combining data across 
test instruments. Aggregation across models was straightforward since 
all three were based on the single underlying construct that the impact 
of a Title I program is determined by subtracting an estimate of how well 
the students would have performed had they not participated in the program 
from 8k measure of how well they performed after participation. In other 
words, the "treatment effect" measured by all models is equal to the 
observed post-treatment performance minus the estimated no- treatment 

performance. ' • . . 

/ 

The three evaluation models differ from one another only in terms 
of the manner in which they generate the estimate of no-treatment performance 
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levels. This methodological difference enables the generation of meaningful 
estimates under a wider variety of conditions than would be possible with 
any single model or any combination of two models. It does not, however, 
produce different kinds of estimates and, if it were possible to find a 
single set of circumstances in which all three models could be employed 
simultaneously, they would all be expected to yield the same estimate of 
the no-treatment performance level. It is this equivalence of outcomes 
which enables the simple aggregation of data across models without any 
need to consider the specific methodological steps each model entails. 

Data can be aggregated across models through simple addition if 
the same test instrument has been used in all cases. The aggregation of 
data from different tests requires additional assumptions and, usually, 
some statistical adjustment. 

The assumptions which underlie the aggregation of data from dif- 
ferent tests relate to the content coverage of the tests. Clearly, if 
test data are to be, aggregated, the tests must measure at least approximately 
the same variable or set of variables. There is ample evidence from the 
Anchor Test Study (Loret, Seder, Bianchini, and Vale, 1974) that the most 
widely used standardized reading achievement tests meet this requirement — 
at least for grades four, five, and six* It is probably safe to assume 
that other standardized, reading tests also "qualify" since visual inspec- 
tion of their content reveals a high degree of similarity. 

There are also dati^'^to suggest that at least one criterion-referenced 
reading test, the Prescriptive Reading Inventory, also correlates suffi- 
ciently highly with standardized reading achievement tests so that aggregation 
presents no problem. This llest, in fact, was found to correlate as highly 
with the California Achievem4jjit Test (one of the instruments included in 
the Anchor Test Study ) as alternate forms of the CAT correlated with one 
another (Roudabush, 1975) . 

It certainly cannot be expected that all reading tests will inter- 
correlate as highly as those mentioned above. The correlation between s 
standardized reading achievement test and a criterion-referenced instrument 
covering a restricted number of objectives or that between two criterion- 
referenced tests encompassing difroTsint objectives will most probably be 
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significantly lower. Even under these circumstances, however , it does 
not appear unreasonable to aggregate achievement data. 

Basically, the assessment of instructional impact is a question 
of measuring gains rather than status. It follows, then, that gains in 
any subset of the skills which comprise reading ability will represent 
an increase in overall ability to read. To use an analogy, it is likely 
that changes in the circumference of subjects' waists may provide nearly 
as sensitive an index of weight gain or loss as actual weighings. To the 
extent that this sort of phenomenon holds true with respect to reading 
ability, it is not necessary to worry excessively about whether one is 
measuring the universe of skills involved or a small sample which varies 
in nearly direct proportionality. Assuming that this argument is sound, 
the only problem which remains is that of converting whatever measures 
are involved to a common metric before the aggregation is made. A pro- 
cedure for accomplishing such a conversion is discussed below. 

The procedure which was developed to express the results from the 
three evaluation models and all test instruments in terms of a common 
metric was simply to express all gains in terms of standard deviation 
units referenced to national population statistics. To accomplish this 
conversion, it is only necessary to subtract from the observed and/or 
predicted posttest score the mean score for the national population of 
the project group's grade-level peers and to divide the difference by 
the population standard deviation. The outcome (s) of these operations 
will always be a score or scores reflecting distance above or below the 
"national norm" quantified in units of the national standard deviation. 

The metric developed in this way does provide a satisfactory 
means for aggregating data which, before transformation, were non-com- 
patible by virtue of reflecting different score distributions. While the 
metric is suitable, however, it is somewhat unesthetic in that the majority 
of obtained scores will be fractional and, in dealing with Title I pupils, 
nearly all will be negative. Td solve the first of these esthetic de- 
ficiencies, it is only necessary to multiply the converted score by some 
constant value. Solving the second requires adding to the product a 
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constant of sufficient size to assure that no negative scores can be 
obtained. 

The choice of specific values for the two constants is entirely 

i 

a matter of convenience. The values chosen weiie 21 and 50, respectively, 
because these values produced a scale with the same range (1 to 99) and 
midpoint (50) as the percentile scale. This feature was considered 
desirable from the viewpoint of interpretability , 

Values on the developed scale were given the name Normal Curve 
Equivalents and quickly became known as NCEs, Similarly, differences 
between observed and expected posttest performance, when expressed in 
terms of this metric, came to be called NCE gains. 

When using standardized test instruments, the conversion of raw 
or scale scores to NCEs does not typically require subtracting the national 
mean from them and dividing the difference by the standard deviation. 
Test publishers' norms tables may be used to translate scores into per- 
centiles and the percentiles can easily be translated into NCEs using 
specially prepared conversion tables (see Appendix F) . 

When objectives- or criterion-referenced tests are used, percen- 
tile conversion tables are most probably not available. This implies that 
it is necessary to know both the mean and the standard deviation of a 
nationally representative sample of the treatment group's grade level 
peers. If these figures are not available as a result of some normative 
data collection effort, they must be estimated. 

Unfortunately, while it appears that good estimates can be made 
of the national standard deviation (see below), there is no good way to 
obtain a reliable estimate of the national mean. This situation precludes 
the possibility of expressing pre- and posttest status in terms of NCE 
scores. Simple algebra, however, shows that the national mean "term" 
drops out of the NCE gain formula, and that NCE gains scores are com- 
puted simply by dividing the difference between observed and predicted 
.posttest performance by the standard deviation of the national distribu- 
tion. Use of unnormed objectives- or criterion-referenced tests thus 
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precludes the aggregation of pretest or posttest NCE status indicators 
but does not preclude the aggregation of NGE^gains with data from norm- 
referenced tests. 

The following paragraphs describe the specific procedures re- 
quired to obtain NCE scores and/or NCE gains for each of the proposed 
evaluation models. 

In Model A, we are dealing with pretest and posttest scores on 
standardized achievement tests. The mean standard pretest and posttest 
scores for any group can be converted to their precentiles using data 
in the test publisher's manual. The percentiles can then be converted 
to NCEs through reference to the conversion table. By subtracting the 
pretest NCE from the posttest NCE, an NCE gain is obtained. This gain 
score is expressed in standard deviation units reflecting nationally 
representative normative data and thus is directly interpretable in 
terms of whatever criterion of educational significance has been estab- 
lished (e.g., one-third of a standard deviation or 7 NCEs.) 
^\ 

In Model Bl, the procedure is similar to that described for 
Model A. In this case, however, the control group posttest mean standard 
score is converted to a percentile and then to an NCE. It is then sub- 
tracted from the treatment group NCE determined in the same way to yield 
an NCE gain. 

In Model B2, the gain is determined from criterion- or objectives- 
referenced tests. Scores on these tests, of course, cannot be converted 
to percentiles with respect to a nationally representative sample. It is 
possible, .however, to obtain an estimated NCE gain if scores are avail- 
able for the treatment group pupils on some standardized achievement 
test. (In our presentation of the model we have assumed that a standard- 
ized test would be administered for needs assessment, purposes.) 

• The procedure for computing the estimated NCE gain is as follows. 
The first step is to subtract the mean posttest score of the control 
group from the mean posttest score of the treatment group. The difference 
is then divided by the standard deviation of the combined treatment and 
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control groups • At this point, the result of the computations is a gain 
score expressed in standard deviation units, but the standard deviation is 
based on the restricted range of abilities of pupils in the treatment 
group rather than on a nationally representative sample. To adjust the 
score so that it estimates gain based on national norms, the procedure 
is to multiply the obtained gain score by the ratio of the treatment 
group's standard deviation on a standardized achievement test to the 
national norm standard deviation on the same test. The score is then 
multiplied by 21 to convert it to an NCE gain comparable to those 
derived form Models A and Bl, 

As mentioned earlier. Model C involves the comp^utation of two 
separate gain scores. One of these gain scores represents the diffei^ence 
between the observed posttest scores of the treatment group and post- 
test scores estimated from the Regression Projection evaluation design. 
The other represents the difference between the intercepts of the treat- 
ment group and comparison group regression lines with the cutoff score 
as determined from the Regression-discontinuity design. The procedures 
for calculating these values are presented in Horst, Tallmadge, and 
Wood (1975); 

If Model CI is employed, the calculated values can be converted 
to percentiles using the test publisher's norms tables. The percentiles 
can then be converted to NCEs using the table included in Appendix F 
and NCE gains can be calculated. 

Model C2, like Model B2, requires obtaining scores of treatment 
group pupils on a standardized achievement test as well as on the 
criterion- or objectives-referenced instrument used for pre- and post- 
testing. The two sets of gain scores described in conjunction with 
Model CI are calculated using raw. scores. The gains are then divided by 
the observed, treatment-group posttest standard deviation. This figure 
is then multiplied by the ratio of the treatment group standard deviation 
to the national norm standard deviation on the standardized achievement 
test as a Model B2, Finally, the result of the previous calculations is 
multiplied by 21 to produce an NCE gain score comparable to those re- 
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suiting from the other models. 

This report does not contain a detailed description of any of the 
three models. The evaluation designs around which the models are built 
are described in Horst, Tallmadge, and Wood (1975) and that document is 
being used as an interim measure until a specially prepared adaptation of 
it can be developed. Data collection, analysis, and reporting forms have 
been developed, however, along with instructions for their use. To 
illustrate these materials. Model A forms through the LEA level are in- 
cluded as Appendix G while Appendix H presents the corresponding in- 
structions. The SEA forms, encompassing all models are contained in 
Appendix I and the associated instructions* in Appendix J. 
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The fourth phase of the study involved the revision and further 
development of the prototype reporting system. In this, phase, the 
models developed in Phase III were takerl to nine State Education Agencies 
so that SEA personnel could contribute their thoughts regarding additional 
revisions to^h^'-system and could advise RMC staff of difficulties they 
mighj. f^ce in implementing such a system. 

The nine SEAs selected jointly by RMC and the U. S. Office of Edu- 
cation were Alabama, Arkansas, Delaware, Hawaii, Michigan, New York, North 
^Dakota, Oregon, and Vermont. These states were chosen on the basis of geo- 
graphic and demographic representation, state testing practices, strength 
of SEA leadership, quality of past State Reports, and the perceived re- 
ceptiveness of the SEA toward the objectives of the study. Each SEA was 
visited by two RMC staff members in late April or early May. During these 
visits, which lasted approximately three days, RMC staff members discussed 
the proposed models in detail with SEA personnel. SEA staff members were 
asked to comment on the feasibility and expense of implementing one or 
more of the models in their states, and notes were made of changes in the 
models suggested by SEA staff. The evaluation and reporting system was 
positively received in most of the nine states. 

After visits had been made to all nine SEAs, a summary was prepared 
of all the criticisms and suggested changes which had come from the SEAs. 
This summary was presented to the Advisory Panel in a fourth meeting of 
that group. Each criticism or suggested revision was discussed by the' 
Panel. and by representatives from RMC and OE until a concensus was reached 
on how best to deal with it. The most significant change which resulted 
from the meeting was that of switching the lowest^J^vel of analysis and 
reporting from project to ^sehbol building. 5y "project", the RMC developer 
had meant a single, identifiable, instructional treatmeat. It was clear, 
however, in talking to SEA personnel that this term had several other 
connotations and would be quite confusing. Furthermore, several advisors 
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felt that even where a district-wide Title I effort was labeled a single 
instructional treatment, implementation differences from school to school 
might be so large that aggregating data across schools would obscure edu- 
cationally significant relationships. 

Other significant changes related to the content of the data report- 
ing forms for Parent Advisory Councils and Personnel and Training. Many 
minor changes were made in the layout of forms and the wording of instruc- 
tions to reduce confusion in analysis and reporting processes. 

After all changes had been made, the revised forms and instructions 
were mailed to the nine SEAs which had been visited for further review. 
Any recommendations received from these SEAs will be taken into account 
when the next revision of the models is made. 

Selection of State Departments of Education to be Visited 

In selecting the nine State Departments of Education which were to 
receive visits from RMC staff members, several criteria were employed. 
First, geographic and demographic characteristics were considered so that 
the selected SEAs would represent nine of the ten U. S. Office of Education 
Regions and so that' they would represent both urban and rural states as 
well as a range of SEA sizes. Other factors which entered into the selection 
are described in the following paragraphs. During this process, records 
of the telephone interviews with State Title I Coordinators and Evaluators 
were consulted^ as were the critical summaries of the State Title I Annual 
Evaluation Reports and the ratings given those reports. 

It seemed reasonable to expect that current testing practices in 
the states would be related to the reception that the evaluation and re- 
porting models would receive. In the telephone interviews, most inter- 
viewees Were asked several questions about tests used for Title I evalua- 
tion in their states. The responses indicated that while standardized 
achievement tests are heavily or exclusively used in most states, several 
states are making extensive use of objectives- or criterion-referenced tests, 
and it was desirable to ensure that at least one state of the nine be com- ' 
mit!ted to objectives- or criterion-referenced testing. The states also were 
si^lected to cover the range of situations with regard to state testing 
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programs, and with regard to the SEAs' expressed willingness to move to- 
ward standardizing LEA evaluation practices. 

The strength of SEA leadership also appeared to be an important 
factor in implementation of the reporting system, and an effort was made 
to include a variety of SEA-LEA relationships in the nine-state sample. 
Specific considerations related to this issue were whether state evalua- 
tions were done internally or by an outside agency, whether the SEA in- 
cluded a large planning and evaluation group, whether the SEA provided 
guidance to LEAs on issues such as needs assessment procedures, whether 
the SEA used a standard format for LEA applications and reports, and 
whether the SEA had computer facilities available for analyzing data. 

The quality oi past reports w^.s also a consideration, and this was 
primarily judged using the ratings given to the reports on the Cognitive 
Benefit, Data Adequacy, and Policy Relevance Scales. The nine states 
selected represented a reasonably wide range of scores on these scales. 

The attitudes of thp SEAs toward the objectives of the study, as 
they were perceived by RMC staff during the telephone interviews, were 
also taken into account. While it was important to present the report- 
ing system to a number of different types of SEAs, it did not appear fruit- 
ful to include in this first sample SEAs who were openly hostile toward 
objective evaluation or toward the purposes of the study. Therefore, all 
nine SEAs selected for visits were judged to be at least sufficiently 
open and receptive so that their comments and suggestions would be con- 
structive. 

After consultation with Office of Education staff, the following 
nine states were selected: Alabama, Arkansas, Delaware, Hawaii, Michigan, 
New York, North Dakota, Oregon, and Vermont. 
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Site Visits to the Selected Nine SEAs 

Visits to the nine SEAs took place in late April and early May. 
Each SEA was visited for approximately three days by one of three two- 
person teams- At least one person at each SEA was familiar with the 
objectives of the project before the site visits were made since, during 
Phase III of the project, SEAs had received a written summary of the pro- 
ject and the critical summaries of their states' reports. A representa- 
tive of each SEA had also participated in a telephone interview with an 
RMC staff member- However, no SEA staff member had been exposed to the 
proposed reporting system prior' to the visit. 

The three teams conducted the site visits in similar fashions, al- 
though the positions and numbers of persons with whom they met varied from 
site to site. The teams usually began the discussion by summarizing the 
work and objectives of the project and then discussing future work plan- 
ned by OE for further development of the reporting system. The teams 
then gave an overview of the reporting system,, similar to that presented 
in the preceding chapter under "The Models." This overview usually led 
to a detailed explanation of NCEs as well as to a discussion of the system- 
atic biases which are introduced into project evaluations by the use of 
grade-equivalent scores. After these issues had been discussed, the re- 
porting forms and instructions were introduced and explained in detail. 

The project- (now building) and LEA-level forms and instructions for 
Model A were generally introduced first, each individual forni being pre- 
sented and discussed item by item. After the first set of forms had been 
discussed, other models were presented in detail only where they differed 
from the first model. The SEA forms for all the models were presented 
last. After SEA personnel had an opportunity to critique the reporting 
system and raise questions, they then filled out some of the impact data 
forms to simulate the reporting process involved in using three of the 
models . 

The data sets included reading achievement scores for Models A, B2, 
and CI. The data were organized so that SEA staff could proceed through 
the reporting system, beginning at the project level, and ultimately com- 
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plete portions of a sample State Report. Working through the data sets 
required close attention to the reporting instructions, and this experience 
enabled SEA staff to identify portions of the instructions which were con- 
fusing or misleading. The process also helped to clarify what would be 
required of persons at each level who would be implementing the models 
and filling out the forms. 

The function of the persons at the SEA to whom these presentations 
were made varied from site to site but usually included the State Title I 
Coordinator and either the Title I Evaluator or a member of the state's 
evaluation group. Other persons with whom the models were discussed at 
one or more sites included members of the state's compensatory education 
department, federal program directors, members of the state's evaluation 
conunittee, outside consultants, and test publishers' local representatives. 
In one state, the KMC team did not meet with any evaluators. In most 
cases, a team would meet with one or two SEA staff members and present the 
models to them. Additional staff members would then be introduced later 
and the presentation would be repeated as many times and in as much detail 
as necessary. 

Discussions of the impact that implementation of the reporting system 
would have on a site arose spontaneously as the models were presented, and 
teams never found it necessary to ask directed questions of SEA personnel. 
At most sites, the models were positively received, although all sites had 
a number of suggestions to make toward improving the models. Several sites 
expressed strong interest in using the reporting system and/or evaluation 
models for state compensatory education programs or Title III programs. 
Several sites also indicated a desire to participate in the field test of 
the reporting system. 

As had been anticipated, the majority of the SEAs were exclusively 
interested in Model A, the Norm-referenced Model." The evaluation methods 
used in this model are more similar to current practices than those used 
in any of the other models, since Model A involves the use of norm-referenced 
tests and does not require testing a control or comparison group. Model A 
also requires only minimal calculations at all levels (means and weighted 
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means,) A few SEAs also expressed an interest in Model C2 (the Special 
Regression Model using criterion- or objectives-referenced tests,) 

Although the SEA personnel appeared to be favorably impressed with 
the proposed reporting system, the site visits revealed several problem., 
with the system as it was originally designed. These problems and their 
solutions are described in the following section. The visits also made 
apparent two difficulties with the reporting system which could not be 
properly solved by revision of the models. The first of these difficul- 
ties arose from the requirement that' a norm-referenced test be administered 
only at times corresponding to real normative data points. This require- 
ment has a major impact on the implementation of Model A, a lesser impact 
on Models Bl and CI, and only a slight impact on Models B2 and C2, 

Some of the inconveniences created by this limitation of testing 
times had been expected. It was anticipated, for example, that most pro- 
jects traditionally administered tests at times which were convenient for 
practical rather than technical reasons and would be reluctant to make 
major changes in either testing schedules or in their choice of tests. 
One problem which had not been foreseen, however, was related to test scor- 
ing services, SEA personnel in states where one or two tests were very 
heavily used were certain that if all projects using one test were to 
administer the test at the same time, the scoring service would be over- 
loaded and many projects would not receive their test scores for several 
months or longer. Discussions with test publishers reinforced this view. 

Although this problem is a serious one, removing the stipulation that 
testing times must correspond to lorming times would compromise the validity 
of evaluation findings based on normative data (Horst, Tallmadge, & Wood, 
1975), One acceptable solution to the problem might be to make linear 
extrapolations from the data points for each test extending several weeks 
in each direction as suggested by Tallmadge and Horst (1974, p, 25) so 
that the test can be used at any point within that time interval. This 
approach, however, requires further .inventigation before it can be advocated 

The second major problem related to the rejection of grade-equiva- 
lent scores as a suitable metric for assessing educational treatment effects 
SEA personnel in .several states expressed a willingness to move away from 
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the use of grade-equivalent scores and indicated that they anticipated no ^ 
great difficulties in doing so, however, others felt that grade-equivalent 
scores were neces,sary to communicate with parents and the public. They 
were skeptical abuu^'"*public^acceptafi^ and understanding of percentiles or 
NCEs. This Issue also requires further investigation. 

* 

Most sites were willing" to make estimates of any additional resources, 
they would need to implement the system. Although there was a wide range 
with respect to the numbers of new personnel SEAs felt they would require 
in order to implement the system, in general the necessary personnel fell 
into two categories. First, some SEAs saw a need for additional clerical 
help in order to maintain records at the "project (now building) level. 
Second, some felt that implementation would initially greatly increase 
the workload of SEA staff responsible for orienting and training local 
Title I personnel, and that additional professional staff might be required. 
Other SEAs felt that this second requirement would be reduced by partici- 
pation in the field test (which would presumably mean that SEAs would re- 
ceive some contractor assistance) , or by a carefully phased implementation. 
In the case of one SEA, the model reporting system was much simpler than 
the system currently in use; thus implementation of the system would not 
require any increase in staffing in that state. Several SEAs expressed 
strong interest in seeing computer programs developed which would organize 
and analyze data in accordance with the models, bel^Leving that such soft- 
ware would substantially reduce the difficulties implementing the system. 
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Revision of the Phase III Models 



Since the visits to nine SEAs had resulted in a number of suggestions 
for revision of the models and had revealed difficulties with the system 
which might be eliminated by revisions, a decision was made to meet with 
OE representatives and with the project's Advisory Panel to discuss possible 
changes in the models. Each criticism or suggestion made by an SEA was 
brought before this group for discussion, rnd decisions were reached on 
whether or not to change the models and what form the chdnge should take. 

. Most of the changes made in the reporting system involved the clari- 
fication of* ambiguous phrases, the addition of more detailed explanations 
and worksheets to the instructions, and the rewriting' of some portions of 
the instructions to make them more explicit. Also several new questions 
were added' to the reporting forms in order to collect additional information 
the Panel thought would be of interest. However, a few larger issues arose 
on the site visits and the Advisory Panel devoted most of the meeting to 
a consideration of these issues. 

One difficulty with the reporting sysfem which became apparent dur- 
ing the site visits was related to the use of the term "project." This 
term, which was used to describe the first level of the reporting system, 
was defined in, different ways by the different states which were visited. 
Since one primary objective of the reporting system was to provide for the 
reporting of project-level information by the states, it was clear that the 
definition of a "project" should be the same for all states, at least as 
far as Title I reporting was concerned. For the reporting system, a project 
was described as a single, identifiable instructional treatment. However, 
discussions with SEA personnel indicated that some states had no clearly 
delineated entities that corresponded to "projects." For this reason, 
the problem could not be solved by the addition of a definition of project, 
nor by the use of some other term such as "activity," "component," or "pro- 
gram. " 

TJie problem seemed to be that in some states, "project" and "LEA" are 
considered identical in terms of applications for funding and evaluation 
reports. At the same time, LEAs frequently provide more than one kind of 



instructional treatment for Title I. The Advisory Panel decided that a 
specific instructional treatment was most likely to be found within a 
single school building, and that the aggregation, of impact data across 
buildings would tend to obscure the effects of the treatment. The Panel 
felt this was true even where several buildings were supposedly using 
the same instructional treatment. Therefore, a decision^, was made to change 
the lowest level of the reporting system from the project level to the 
building l^vel. 

The second major problem which was considered by the Advisory Panel 
was the objection . several states raised to the inclusion of information 
on Parent Advisory Councils and Personnel and Training in the reporting 
system. Although some states felt that neither type of information was 
appropriate in an evaluation report , objections seemed to focus on the 
reporting forms related to Parent Advisory Councils. Several states felt 
that the information they collected in their monitoring activities would 
be more accurate than the information that would be reported by schools 
and districts. These states felt that each SEA should report on the re- 
sults of its monitoring rather than simply aggregate possibly inaccurate - 
reports from LEAs- On the other hand, the parent representatives on the 
Advisory Panel felt that the PAC reporting forms at each level served an 
important function and should be retained. The Panel eventually agreed 
that no final decision should be made on these forms until all SEAs had 
had an opportunity to give their opinions on the issue. 

When all the necessary changes had been made in the forms and in- 
structions, copies were sent to each of the nine SEAs which had been 
visited so that SEA staff could make any additional comments on, or criti- 
cisms of, the reporting forms and instructions. The suggestions they 
make will be taken into account when the reporting system undergoes its 
next revision. 
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VI. CONCLUSIONS AND RECOMMENDATIONS 



The analyses of State Title I Annual Evaluation Reports conducted 
in the first two phases of the study revealed that the variety of current 
state reporting practices made aggregation of any data across all states 
impossible. Although there were some reporting formats, notably in the 
area of participation, which were quite commonly used and which presented 
policy-relevant information-, it seemed clear that only the standardization 
of state reporting practices would make it possible to obtain an accurate 
national picture of Title I. The model reporting system designed in the 
last two phases of this study was intended to make minimal demands on state 
and local personnel while providing policy-relevant information to decision 
makers in Washington. Based on reactions to the reporting system thus far, 
it is not unreasonable to expect that the system will be adopted by a 
majority of states, even in the absence of a legislative mandate. 

Information obtained during the study indicates that the portion of 
the system related to the collection and reporting of impact data may cause 
problems in many states. The difficulties stem from the fact that it is 
possible to collect reliable and valid impact data only through strict 
adherence to sound educational measurement and evaluation design practices. 
In the past, local Title I evaluators have unwittingly engaged in practices 
which have compromised the results of their evaluations. Thus, the model 
reporting system has necessarily included guidelines and requirements for 
conducting valid assessments of program impact. Among these requirements 
are (a) norm-referenced tests must be administered at times for which the 
publishers have real normative data points; (b) whan objectives- or criterion- 
referenced tests are employed, the evaluation design must make use of a 
control or comparison group; ana (c) grade-equivalent scores cannot be used 
to measure project impact. Mqst local Title I evaluators will have to 
change their evaluation procedures to meet these requirements, and making 
these changes will cause them some inconvenience. 

There are several ways in which difficulties related to the collec- 
tion of valid impact data can be mitigated, and there are also a number of 
questions related to this area of the system design which should be answered 
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before the system can be considered complete. This study was the first part 
of a larger effort devoted to the design, field testing, and eventual im- 
plementation of a Title I reporting system. Thus, while a number of issues 
related to the reporting and evaluation system remain unresolved, and sev- 
eral steps could be taken to make the system easier to adopt, work has be- 
gun under a new contract to address these problems. The areas in which 
further work or investigation are needed will be described briefly in this 
chapter. 

The model reporting system currently consists of sets of forms and 
related instructions. Descriptions of the models and requirements for their 
implementation have thus far been communicated orally to state and local 
personnel, but there is a clear need for a written document related to model 
selection and implementation. This document will enable state and local 
personnel to choose the evaluation model most appropriate to their needs, 
and will then inform them of the steps necessary to implement the model at 
a time when they are able to make plans to meet all the model requirements. 
As it is now conceived, this document will be quite similar to A practical 
guide to measuring project impact on student achievement (Horst, Tallmadge, 
& Wood, 1975) but will be directly tied to the model reporting system. 

The production of computer software for each of the models at all 
levels will remove much of the computational and clerical burden from the 
reporting system and will make the system easier to use and thus more at- 
tractive than data collection systems currently used by most states. Com- 
puterizing the system will reduce paperwork at the LEA and SEA levels and 
will probably result in decreased errors, as well as ensuring that all data 
in the reports will, in fact, be produced in a standard manner. Unfortunately, 
it appears that in the smaller or less-populated states, use of a computer 
to store and analyze Title I data would not be cost effective, particularly 
at the LEA level. Thus, although several larger states strongly recommend 
the development of computer programs, this step will probably not provide 
most rural states an incentive to adopt the system. 

The requirement that norm-referenced tests be administered only at 
times for which real normative data are available makes the use of tests 
having only one data point inconvenient, and in some instances , impossible, 
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Such tests cannot be used for fall pretesting and spring posttesting ex- 
cept at grades 4, 5, and 6 for tests included in the Anchor Test Study 
(Loret, et at., 1974). Effort will be devoted to persuading test pub- 
lishers to produce two or more empirical data points for their norm- 
referenced tests. There is reason to believe that' some test publishers 
are now considering taking this step, and at least one publisher already 
intends to produce a second data point. Of course, publishers' interest in 
producing additional empirical data points will depend to some extent 
on how many states plan to adopt the model reporting system. 

Appropriate use of any norm-referenced test could be made easier 
by the development of extrapolated norms tables where percentile values 
would be read out as a function of the test score and the testing date. 
These double-entry tables would make it possible for local evaluators to 
administer a test at any date over a several-month period without intro- 
ducing systematic biases into the data. If such tables were made available 
for the commonly used tests. Title I evaluators could administer tests at 
times more convenient for them than the exact norming dates. Since these 
administration times presumably would vary among the projects, the problem ^ 
of overloading the scoring services would be somewhat reduced, and test 
results would be. returned to the projects more quickly. If the test used 
had more than one normative data point, and if projections were produced 
for each data point, the test could be administered at any f a number of 
times during the school year. 

Model A permits the use of any test for which adequate norms are 
available. However, no criteria have been developed for judging the ade- 
quacy of norms, nor has sufficient information been collected concerning 
various standardized tests to permit a determination of which standardized 
tests have norms appropriate for use in Model A. Additional information 
must be collected from test publishers before this issue can be resolved. 

Difficulties created by out-of -level testing cause serious problems 
for Title I evaluators, since most Title I students cannot appropriately 
be tested using measures designed for their grade level. It seems clear 
that bel ow— level tests must be used to avoid floor effects, but the issue 
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of out-of-level testing needs further investigation. Information is needed 
on l^bw to judge which level of a test to administer to a group of students 
and/ what tests have adequate between-level articulation so that they can 
be used out of level. There is also a need to work with test publishers 
to produce better norms tables so that it will be possible to convert raw 
scores on an out-of-level test directly to within-level percentiles. 

Model A is expected to be the most widely adopted model since the 
other models require testing control or comparison groups and such testing 
cannot presently be paid for by Title I funds. However, a question has 
arisen about the effects of regression toward the mean on the posttest ex- 
pectation used in Model A. In this model, the pretest percentile is used 
as the posttest expectation under the assumption that without special treat- 
ment a Title I student would maintain his position with respect to his peers. 
However, work needs to be done in order to determine whether this assumption 
should be tempered to take regression effects into account. 

Although Model B is the most precise of the models, it is extremely 
difficult to implement, since control groups for Title I students are not 
normally available. The possibility remains that this model could be used 
with a non-comparable control group (e.g., low achieving students in a non- 
Title I school), but further investigation is needed to determine how dif- 
ferent the control group can be from the Title I group before the additional 
precision of this model is lost. 

Aside from these issues, further revision of the forms and instructions 
will be necessary a3 more state and local personnel have the opportunity to 
make inputs into the system. Also, the system must eventually include 
criteria for determining which individual building-level reports should 
be included in the State Report to the Office of Education. As states 
decide to implement the system, the documents will need to be tailored so 
as to correspond to the particular needs of the state while maintaining their 
usefulness at the national level. As revisions are made, they will incor- 
porate any new information resulting from investigation of the issues dis- 
cussed here, t . 
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APPENDIX A 

State Reports which were Analyzed 
for the Years 1969-74 
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State Reports Analyzed 
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STATE .. YEAR ^SCORE 



--^CALE OF ADEQUACY MD VALIDITY OF REPORTED DATA Reviewer 



COGNITIVE BENEFIT INFORMATION 

2.0 Cognitive data are ideally* presented for all or a representative (or 
random) sample of students served. 

1.5 Cognitive data are ideally* presented for a probably representative sample 
of students (description of sample or sampling procedure is not adequate for 
proof of representativeness) . 

1.0 Cognitive data are ideally* presented for a possibly biased sample of students 
(description of sample or sampling procedure does not suggest that biases 
exist; neither, however, is there any evidence to suggest that it is 
representative). * 

0.5 Cognitive data are ideally* presented for a probably biased sample of 

students (description of sample or sampling procedure suggests that the sub- 
groups in the population served are likely not to be represented proportion- 
ately to their size. There is no evidence, however, that the sample was 
selected so as to show the largest cognitive benefits) . 

0. 0 No hard cognitive data are presented. 

* Cognitive benefit data ideally presented means: 

1. Pre and posttest data £r gain scores for treatment group, or posttest data for 
treatment and control group. (Deduct 0.3 if absent) 

2. Measures describe distribution of scores (standard deviations, percentiles, 
deciles) not just measures of central tendancy (mean, median, mode) — Deduct 
0.2 for use of non-standard measures* 

4. Tests administered and scored by independent evaluator. (Deduct 0.1 if done 
by classroom teachers or other "involved" personnel) 

"^5. Presentation is based directly and solely on achievement test data. (Deduct 
0.3. for contaminated data) 

TARGET GROUP INFORMATION ' 
Add to Cognitive Benefit score: 
0.2 for total number served 

0.2 for breakdown of target group by grade level 
0.2 for breakdown of target group by rate 

0.2 fp-r^Breakdown of target group by urban-rural (or community size) 
Qv2 for breakdown of target group by socioeconomic status 
0.2 for any multidimensional breakdown 
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IF COGNITIVE BENEFIT SCORE IS > 0.7 
AND 



IF TARGET GROUP DATA ARE INTEGRATED WITH COGNITIVE DATA 
THEN 

DOUBLE TARGET GROUP SCORE 

PROGRAM INFORMATION 

Add to composite score: 

0.2 for list of programs offered 
0.2 for breakdown of number served by program 
0.2 for breakdown of programs by grade level 
0.2 for any multidimensional breakdown 

IF COGNITIVE BENEFIT SCORE IS > 0.7 
AND 

IF PROGRAM DATA ARE INTEGRATED WITH COGNITIVE DATA 
THEN 

DOUBLE PROGRAM SCORE 

COST INFORMATION 

Add to composite score 

0.2 for total or per-pupil costs 
0.2 for breakdown of costs by program 
0.2 for breakdown of costs by grade level 
0.2 for any multidimensional breakdown 

IF COGNITIVE BENEFIT SCORE IS > 0.7 
AND 

IF COST DATA ARE INTEGRATED WITH COGNITIVE DATA 
THEN 

DOUBLE COST SCORE 
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READABILITY SCALE 



Total 
Possible 



1. Abstract or Synopsis of Report ; Report includes abstract or synopsis 
at the beginning. 

(1 point = Most of the major findings are briefly discussed; .5 points = 
Some of the major findings are briefly discussed, or the types of 
information included in the report are described; 0 points = No ab- 
stract is given.) 

2. Organization ; There is a table of contents, and the report is organized 
into sections which actually cover the materials listed in the table of 
contents. 

3. Integration ; Repoirt includes either the general data across the entire 
state or summarizes information about smaller units such as LEAs, projects, 
or programs. 

4. Writing Style ; Report is written in a clear and unambiguous manner. 
"As a result, it tells exactly what you mean and reduces the chance 
that readers xvill misunderstand [Shaffer, 1967]." The writing is not 
excessively repetitious or convoluted. 

(2 points = Most of the report meets the above criteria; 1 point = Only 
some parts of the report meet these criteria; 0 points = Most of the 
report does not meet the above criteria.) 

5. Technical Presentation ; Descriptions of methods and results are complete 
and explicit. t 

(2 points = Most of the descriptions have complete and explicit ex- 
planations; 1 point = Most sections of the report present minimal ex- 
planations; 0 pointy = Most sections of the report do not have complete 
and explicit explanations.) 

6. Tables and Figures ; Text refers to specific tables and figures which 
are close to the point where they are discussed. The tables are clearly 
labeled and easily readable. The figures, including the coordinates, are 
clearly labeled and easily readable. 

(2 points = Most o£ the tables and figures fulfill the above criteria; 
1 point = Some of the tables and figures meet the above criteria; 
0 points = Most of the tables and figures do not meet these criteria.) 

7. Summary or Conclusion ; Statements about the major points described in 
the report are presented. 

(1 point = The results are summarized. They are discussed or recom- 
mendations are made; .5 points = The results are only summarized; 
0 points = No results or discussion is presented.) 



1 8. Copying ; Materials are clear and easy to read. 
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state 



Year 



Score 



Reviewer 



SCALE OF POLICY RELEVANCE 

Achievement Benefit Information 

4 = Achievement test data; pre, post, and gain by project and aggregated across 
state 

3 = Data across state plus achievement data for exemplary projects or no state 
data, but adequate project information 

2 = Data across state, no project level information 

1 = Inadequate statewide aggregation (anectodal information) 

0 = No achievement data 
Resource Allocation 

3 = Project level breakdown of expenditure by service area and statewide aggregation 

2 » Statewide breakdown of expenditure by service area, or sufficient project level 

breakdown 

1 = Inadequate statewide aggregation (anecdotal) 

0 = No data on expenditures 

'"''Add 1 point for information on numbers eligible and numbers served. 
Needs Assessment 

4 » Project level descriptions of needs assessment procedures and results plus 

statewide summary 

3 » Project level descriptions of needs assessment procedures and results; no 

statewide summary 

2 • Statewide summary without project level description 

1 - Inadequate statewide aggregation 

0 " No needs assessment data 
Public/Non-public 

2 • Breakdown of expenditures and number of participants by county plus statewide 

aggregation 

1 = Statewide breakdown 
0 * No breakdown 

* 
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Parent Advisory Councils 



2 - School or district level descriptions of PAC membership composition, activities 
and achievements plus statewide summary 

1 ~ Statewide summary 

0 = No information 

Other Policy Issues 

Add one-half point for each policy issue addressed beyond those mentioned up to 
a maximum of four. Specify the issues addressed: 

Comparability 
. Instructional time 
Time of intervention 

Integration with other federal and/or compensatory programs 
Role of SEA 
Late funding 
Other 



Other^ 
Other_ 
Other 
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Letter Sent to State Title I Coordinators 
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February 29, 1975 



Title I Coordinator 
State Office Building 

1 Main Street ^ ^ 

Harrisford, Calissippi GOOOl 

Dear Coordinator: ' 

Since 1 July 1974, KMC Research Corporation has been working under 
contract with U.S.O.E.'s Office of Planning, Budgeting, and Evaluation 
on a project concerned wijih improving the State ESEA Title I evaluatj.on 
and reporting system. In conjunction with this project, a member of our 
staff will be contacting, you shortly to solicit your ideas on some of 
the more important issues . 

■If you attended one of the Regional Title I Coordinator meetings 
held at Atlanta and Dallas at the end of September and San Francisco at 
the beginning of October, you may have heard my brief presentation de- 
scribing our study . In case you did not hear the presentation, or to 
refresh your memory if you did, I have appended a brief project summary. 
The project, as I am sure you recognize, is intended to fulfill some of 
the legislative mandates contained in the recently enacted Educational 
Amendments of 1974 (Public Law 93-380). 

During Phases I and II of our study we examined past Title I reportis 
f,rom 1971, 72, and 73 in terms of their content, the adequacy of the data 
they presented, their readability, and the extent to which they addressed 
policy issues which are currently thought to be relevant to federal- level 
decision making. We then wrote brief critical summaries of each report. 
Copies of the summaries of reports from your state are also appended. 

The ratings and critical summaries which we made of past reports 
are not to be interpreted as evaluations of any state*s performance since 
ho guidelines were provided prior to report preparation indicating the 
content and format desires of U.S.O.E. Instead of providing ah index 
of performance, our ratings and comments are useful only in indicating 
how far past reports have deviated from some probably unattainable and 
not yet completely defined ideal. 

During Phase III of our study we will be trying to identify realistic 
objectives with respect to evaluation practices and reporting content 
and format which will improve the comparability of reports across states 
and enable tht* aggregation of data to provide a nation-wide picture of 
Title I operation and impact. In addition to working closely with our 
Advisory Panel whose members represent state and local education aggsncies, 
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the Council of Chief State School Officers, the National Advisory Council 
on the Education of Disadvantaged Children^ and parents of Title I children, 
our plans call for telephone conversations with the Title I Coordinators 
and/pr their designates from each state. Our objectives in these tele- 
phone discussions will be to solicit specific comments and suggestions 
relevant to the design of an improved Title I evaluation and reporting 
system. 

A member of our project staff will call you approximately two weeks 
from the date of this, letter. In addition to any general comments you 
may wish to volunteer, we will want to explore the following issues with 
you: 

1. Is it feasible, and if so how large a burden would it place 
on sea's, to report Title I participation 

a) in terms of unduplicated counts of children served 
by grade level in public and nonpublic schools sepa- 
rately for summer and regular school sessions, and 

b) in terms of the number of children receiving each 
type of program service by grade level in public and 
nonpublic schools separately for summer and regular 
school sessions? 



2. Is it feasible, and if so how large a burden would it place 
on sea's, to report total state dollar allocations and 
expenditures, including administrative funds with separate 
account/lng for carry-over funds; funds for handicapped, 
delinquent, and migrant children; and parts B and C of ESEA 
Title I? 

3. Is there any reasonable way of separating expenditures 
according to public vs. nonpublic school participants? 

4. Can total and per-pupil expenditures be broken down by type 
of program service separately for summer and regular school 
sessions? 

5. Can expenditures be broken down according to standard 
accounting categories? 

6. Would it be feasible to impose standardized evaluation prac- 
tices on lea's for the assessment of achievement benefits 
including the specification of acceptable standardized test 
instruments and testing times? ^ 

7. Would it be feasible to report test score data by grade level 
by project — assuming only a minimum of information was desired 
at the project level (such as mean pretest score, mean post- 
test score, and the number of children tested)? 



175 



194 



page three 



8. Would it be feasible to adopt standard needs assessment practices 
state^wide and subsequently report on compliance with these pro- 
cedures at the project level? 

9. Would it be feasible to adopt standard practices with respect 
to the composition, terms of office, and activities of parent 
advisory panels and subsequently report on compliance with 
these practices at the project level? 

Another concern of ours relates to the use of grade-equivalent 
scores and norm-referenced evaluation models for assessing cognitive 
achievement benefits. These issues are complex and cannot be adequately 
discussed here or by telephone. You should recently have received, how- 
ever, a report entitled Measuring Achievement G ains in Educational Projects 
authored by Horst, Tallmadge, and Wood. This report deals extensively 
with the problems associated with grade-equivalent scores and norm-refer- 
enced evaluation models (see especially Hazards 1 and 3, pages 9 and 13). 
If you have time to read some of this, material, we would very much like 
to obtain your reactions. 

I apologize for the fact that we are asking for so much of your time. 
Without your help, however, we can have no hope of designing a reporting 
system which both is workable and meets the information needs of educa- 
tional policy makers at the federal level. We will be very grateful for 
whatever time you can devote to thinking about the issues I have described 
and communicating your thoughts to us. 

Sincerely yours. 



G. Kasten Tallmadge, Director 
Learning Systems Group 
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Xile to Normal Curve Equivalent Conversion Table 



*ile 








Xile 


NCI 


1 


1 


36 


,42 


71 


62 


2 


7 


37 


43 


72 


62 


Q 
J 




Jo 


44 


\73 


63 


/. 


IJ 


39 


44 





64 


3 


13 


40 


45 


75 


64 


6 


17 


41 


45 


76 


65 


. 7 


19 


42 


46 


77 


66 


o 
o 




/ o 

43 


46 


78 


66 


9 




44 


47 


79 


67 


10 


o o 

23 


45 


47 


80 


68 


11 


24 


46 


48 


81 


68 


12 


25 


47 


48 


82 


69 


13 


26 


48 


49 


83 


70 


14 


o 

27 


49 


49 


84 


71 


15 


oo 

28 


50 


50 


85 


72 


16 


29 


51 


51 


86 


73 


17 


30 


52 


51, 


87 • 


74 


18 


o t 

31 


53 


52,' 


88 


75 


19 


o o 

32 


54 


52 


89 


76 


OA 

20 


o o 

3^ 


55 


53 


90 


77 


21 


33 


56 


53 


91 


78 


22 


34 


57 


54 


92 


80 


23 


o / 

34 


58 


54 


93 


81 


24 


o e 

35 


59 


55 


94 


83 


25 


36 


60 


55 


95 


85 




JO 


61 


56 


96 


87 


27 


37 


62 


56 


97 


90 


28 


38 


63 


57 


98 


93 


29 


38 


64 


58 


99 


99 


30 


39 


65 


58 






31 


40 


66 


59 






32 


40 


67 


59 






33 


41 


68 


60 






34 


41 


69 


60 






35- 


42 


70 


61 
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GENERAL INFORMATION 

LEA Name or Niimber ^ 

Building Name or Number 

□ Regular school year 

□ Summer 

□ Yearlong 

Hours of student project exposure per week 

Total weeks of project duration 

Instructor : pupil ratio : 

Total project cost 

Title I funds Other supplementary funds 



PARTICIPATION 

Number of Children 

Grade Levels (Unduplicated Count) 

Served Public Nonpublic Total 

' Pre K ^ 

- K 

1 

2 

3 

4 . 

5 

6 

7 

8 

9 

10 . 

11 

12 

Total . " 



Number of Children Served , 

Program Components (Duplicated Count) 

Reading/Lang. Arts/Communication 

Math 

Other Academic 

Affective 

Support Services 
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PERSONNEL AND TRAINING page 2_ of 

Building Name or Number • 



Number and Type of Personnel Eiiq)loyed in the Project 

Type of Personnel Number (Full Time Equivalent) 
Instructional 

Teachers 

Instructional Aides or Assistants 

Reading Specialists 

Mathematics Specialists 

Directors or Coordinators 

Resource Personnel 

Other Instructional Personnel 

(specify) 



Supportive 

Counselors or Psychologists 

Other Supportive Personnel 
(specify) 



Does the Title I project have preservice . and/or inservice training sessions 
for either teachers or aides? Yes No 



Check* the Topics Covered during Training 
Topic 

Orientation to Title I project 

Project Planning and Design 

Subject Matter Areas (Reading, 
Math, Language Development, etc.) 

Use and Duties of Teacher Aides 

Use of Supportive Services 
(Counseling, Medical , etc. ) 

Dissemination of Research Results 

Evaluation Design 

Other 



Teachers 


Aides 


□ 


□ 


□ 


□ 


□ 


□ 




■ □ 




□ . 


□ 


□ 


□ 


□ 


□ 


□ 




Check the Range of Days of Training Received by Teachers and by Aides 

Days Teacher Training Aide Training 

Fewer than 1 C] C] 

1-2 □ □' 

• 3-5 □ □ 

6-10 □ □ 

More than 10 [U ^- C] 
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BUILDING LEVEL PARENT ADVISORY COUNCILS 



LEA Name or Number 

Q Building Level PAC 

Building Name or Number 
n District Level PAC 

District Name or Number 



How many members are there on the PAC? 



How many members are parents of children who are enrolled in Title I 



programs? 



Check each activity in which the PAC participated. 

C Used needs assessment information to make recommendations about 
the most pressing needs which should be concentrated upon by 
Title I programs. 

□ Reviewed periodic progress reports. 

□ Reviewed evaluations of current and previous Title I projects. 

□ Informed and consulted parents about Title I services. 

□ Advised the LEA about the Title I application. 
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page 



of 
A 



Building Name or Number 

Grade 

ACHIEVEMENT DATA 

G Reading 

□ Mathematics 

Name of Pretest- 

Level Form 

Norms used 



Name of Posttest 

Level Form 

Norms used 



Date 



Make-up Date 



mo/day/yr 



mo/day/yr 



Date 



Make-up Date 



mo/day/yr 



mo/day/yr 



Name or ID Number 


Raw Score 


Standard Score 


Pre 


Post 


Pre 


Post 


1. 










2. 










3. 










4. 










5. 










6. 










7 . 










8. 










9. 










10. 










11. 










12. 










13. 










14. ' 










15. 










16. 










17. 










18. 










19. 










20. 










21. 










22. 










23. 










24. 










25. 
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page of 

A 



IMPACT DATA 

Building Name or Number 

READING 

Grade 

Name of Pretest '_ 

Level Form Date Make-up Date 

mo/day/yr . mo/day/yr 

Norms used 

Name of Post test • 

Level Form Date Make-up Date 

mo/day/yr mo/day/yr 
Norms used 

Wliat percent of the project occurred between the pretest and the posttest? _ 

Number of children served in this grade level in reading 

Number wi^th both pre- and posttest scores ^ 

Standard score means: Pretest Posttest ^_ 

Percentile equivalents: Pretest Posttest \ 

If number of children with pre- and posttest scores is different from number 
of children served, please explain. 



MATHEMATICS 

Grade 

Name of Pretest . 

Level Form Date Make-up Date 

mo/day/yr mo/day/yr 
Norms used 

Name of Posttest 

Level Form Date t-lake-up Date 

mo/day/yr mo/day/yr 

Norms used 

What percent of the project occurred between the pretest and the posttest? _ 

Number of children served in this grade level in mathematics 

Number with both pre- and posttest scores 

Standard score means: Pretest Posttest 

Percentile equivalents: Pretest Posttest 

If number of children with pre- and posttest scores is different from number 
of children served, please explain. 
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GENERAL INFORMATION 

LEA Name or Number . 

Number of buildings with Title I projects in LEA 

COST • 

Total Cost (for all projects in LEA) 

Title I Cost Other 

PARTICIPATION (for all projects in LEA) 

Number of Children 

Grade Levels , (Unduplicated Count) 

Served Public Nonpublic Total 

Pre K 

K 

1 ' 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

Total 



Number of Children Served 
Program Components (Duplicated Count) 

Reading/Lang. Arts /Communication 

Math 

Other Academic 

Affective 

Support Services / 

/ 

EVALUATION 

Number of buildings submitting achievement test data 

If number of buildings reporting achievement test data is different from the 
total number of Title I buildings in LEA, pleas^ explain. 



Number of buildings using each evaluation model: 

Model A Model Bl Model B2 Model CI Model C2 
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DISTRICT LEVEL PARENT ADVISORY COUNCILS 



LEA Name or Number 

□ 



□ 



Building Level PAC 

Building Name or Number 
District Level PAC 

District Name or Number 



How many members are there on the PAC? 

How many members are parents of children who are enrolled in Title I 
programs? 

Check each activity in which the PAC participated. 

I I Used needs assessment information to make recommdations about 
the most pressing needs which should be concentrated upon by 
Title I programs. 

CD Reviewed periodic progress reports. 

I I Reviewed evaluations of current and previous Title I projects. 
I I Informed and consulted parents about Title I services. 

I I Advised the LEA about the TitlevI application. 

\ 
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PARENT ADVISORY COUNCIL INFORMATION page of ^ 

LEA Name or' Number [ - 

Number of Title I participating buildings in the LEA^ 

Number of building-level PACs ^ 

If there is a difference between the number of buildings and the number of 
building-level PACs, please explain: 



Does your LEA have a district-wide PAC? 
Yes No 



If no, please explain: 



Number of PACs (including district-level PACs) that have a majority of 
members who are parents of Title I children 

Giv<^the total number of PACs which engaged in the following activities: 



Number 

Used needs assessment information to make recommendations about 

the most pressing needs which should be concentrated upon by 
Title I programs. 

Reviewed periodic progress reports'. 

Reviewed evaluations of current and previous Title I projects. 

Informed and consulted parents about Title I services. 

Advised the LEA about the Title I application. 

Place a check mark before each type of information and/or assistance furnished 
to the PACs by the LEA: 

Q Copies of the Title I Act, Federal and State regulations and guidelines. 
Copies of the LEA's current application. 
Progress reports of Title I projects . 

□ Orientation and training materials. 
CD Plans for future Title I projects. 

n Needs assessment data from Title I projects. 
[U Evaluation results of prior Title I projects, 

□ Procedures -for promptly responding to complaints and suggestions from 
parents. ^ 

CU Funds provided to' promote PAC activities. 
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PERSONNEL AND TRAINING page of 

LEA Name or Number , 

Number and Type of Personnel Employed in All Projects ^ 

Type of Personnel . Number (Full Time Equivalent) 
Instructional 

Teachers v. . " , 

Instructional Aides or Assistants 

Reading Specialists . 

Mathematics Specialists . . - 

Directors or Coordinators 

Resource Personnel 

Other Instructional Personnel 

(specify) 



Supportive 

Counselors or Psychologists 

Other Supportive Personnel 
(specify) ■ 



Number of Buildings whose Personnel Rece4.ved Training in Specified Topics 

Teachers Aides 

Topic (Number of Buildings) (Number of Buildings) 

Orientation to Title I Project . 

Project Planning and Design 

Subject Matter Areas (Reading, 

Math, Language Development, etc.) ^ ■ . 

Use and Duties of Teacher Aides 

Use of Supportive Services 

(Counseling, Medical, etc.) ' 

Dissemination of Research Results 

Evaluation Design 

Other 



Number of Buildings whose Personnel Received Training within Specified Ranges 

of Days . ^ 

Teacher Training ^ Aide Training 

Days (Number of Buildings) (Number of Buildings) 

Fewer than 1 . ♦ 

1-2 \ 

3- 5 ^ 

6-10 

More than 10 
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LEA Name or Number 

Building Name or Number 

□ Regular school year 

□ Summer 

□ Yearlong 

Hours of student project exposure per week 

Total hours per project 

Instructor :pupil ratio : 

Number of children served (unduplicated) _ 

Total project cost . 

Cost per pupil 

ACHIEVEMENT DATA 

□ Reading 

□ Mathematics 



Grade 


N 


Percentile Score 


Normal Curve 
Equivalent 


Normal 
Curve 
Equivalent 
Gain 


Pre 


Post 


Pre 


Post 


Pre K 














K 














1 














2 














3 














4 














• 5 














6 














7 














8 














7-8 














9 














10 














11 














12 














9-12 




1 
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f Instructions for 

Building Level Evaluation Report 

Orientation to the Reporting Forms 

Each form to be filled out for the building has a form number cen- 
tered at the bottom of the page. The form number will begin with an R 
(for report) or a W (for worksheet). Forms with R-numbers are to be 
sent on to the LEA as part of the building report. ^ Forms with W-numbers 
are provided for the convenience of building staff and will not normally 
be sent to the LEA. Worksheets are shaded along the right margin so 
that they can be easily separated from report forms. Form nximbers in a 
given model will not necessarily be consecutive. 

At the upper right of some forms, under the spaces for page 
numbering, forms which are ^specif ic to one evaluation model are identi- 
fied according to model (A, Bl, B2, CI, or C2) . Forms which do not show 
this identification are general to all models. These forms should be 
completed for the building even if none of the evaluation models apply 
to the Title I project. (These general forms are R-1 and R-2.) 

General Information (Form R-1) 

Form R-1 should be filled out regardless of the methods used to 
evaluate your project. The name or number of the LEA and the name and/or 
number of the building should appear on the first two lines. The time 
of year during which the project is conducted should be checked. 

^ The average number of hours that a child spends each week in 

the project should be noted as well as the total number of weeks between 
the project's beginning and end. 

If the project has teachers, aides, pupil's peers, or older 
children acting as instructors, they should be counted as instructors 
when calculating the instructor :pupil ratio. In general, anyone regard- 
less of age or training, who functions in an instructional capacity 
should be counted as an instructor* In calculating the ratio, consider 
only the time during which the student is actually participating in the 
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project. For example, if a project consists of an aide tutoring each 
child individually for 30 minutes a day, the instructor : pupil ratio is 
1:1 although the aide may tutor several . children each day. 

All money spent on project children which is above and beyond 
the per-pupil expenditure for the regular school program should be in- 
cluded in computing the total project cost for the building. Title I 
monies and funds from any oth^r source such as State Compensatory Educa- 
tion programs which are used to support the project should be included. 
The total should include the salaries of all full time project personnel, 
the appropriate proportions of the salaries of part-time personnel, the 
fixed costs associated with these salaries, supplies and materials, and 
the pro rata share of capital equipment costs based on useful life 
expectancies. » In the blanks provided on the form, enter the amount of 
project costs which is paid by Title I, and the amount which is paid for 
out of other supplementary funds. 

Participation 

An unduplicated count should be made of the number of public 
and nonpublic school children served at each grade level. Each child 
in the project is counted only once. If the project includes children 
in an ungraded classification, use the following table to assign the 
child a grade classification on the basis of his age for the purposes 
of this report. 



Age-to-Grade Conversion Table 



Age 
Grade 


<5.0 

Pre K 


5-5.11 

K 


6-6.11 

1 


7-7.11 

2 


8-8.11 
3 


9-9.11 

4 


10-10.11 
5 


Age 

Grade, 


11-11.11 
6 


12-12.11 
7 


13-13.11 
8 


14-14.11 
9. 


15-15.11 
10 


16-16.11 

11 


17-17.11 

12 



Provide a duplicated count of the number of children served in 
each program area. In this case a child may be counted once for each 
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program area in which he participates. For exanqple, if a child receives 
instruction in both Reading and Mathematics he is counted twice, once 
under each program component. 

Pers.onnel and Training (Form R-2) 

Form R-2 should be completed regardless of the methods used to 
evaluate the project. Enter the name or number of the building on the 
top line of the form. Then indicate the number of salaried persons of 
each type employed by the project in terms of full-time equivalents. 
For instance, two half-time aides would appear on this form as one aide. 

Mark the appropriate answer to the first question in the middle 
section of the form. Then mark the topics covered during training for 
teachers and or aides, marking as many topics as apply. For days of 
training, a range should be checked for both teachers and aides if both 
received training as part of the Title I project. 

Achievement D^ta XForms W-1 and R-3) 

A worksheet for recording achievement test scores (Form W-1) 
has been provided. A set of these worksheets should be filled out for 
project participants in each grade level for Reading and/or Mathematics. 
The reading test scores for a grade leivel will be recorded on one set 
of sheets and the mathematics test scores for that grade level on a 
second set of sheets. 

Write the name or number of the building at the top of the 
worksheet. Because more than one page may be required to record the 
scores of all participants in a grade, the grade level group should be 
clearly identified at the top of each page. This will simplify identi- 
fying a particular subset of data when it becomes part of a larger set. 
Then check whether the scores represent reading or mathematics data. 
The complete name or the pretest and posttest must be listed as well as 
the level and form of the tests and the norms used in scoring the test. 
Testing dates and make-up dates are critical in norm-referenced evaluation 
and should be carefully recorded. Make-up tests should be administered 
within two weeks of the original test. 
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List all of the names of the students in the project or, if 
preferred, list the identity number of each student in the far left 
column. 

Record the raw pretest scores in the appropriate column and 
look up the corresponding standard scores (scaled scores , converted 
scores, or expanded standard scores are other names for the same type 
of score) using the test publisher's tables. Enter these scores in the 
appropria'te column. After the posttest is given, the same procedures 
should be followed. 

\ After testing is completed for the project, draw a line through 
the name of any child who does not have both pre- and posttest scores. 
A brief explanation of the reason the score is missing should be noted 
in the blank. His data will not be included in further computations. 

The worksheets normally will be kept at the building and will 
not be sent on to the LEA. Information on achievement test "scores that 
will be sent to the LEA is recorded on Forms R-3. One of these sheets 
should be filled out for each grade level that the project serves. The 
building name or number, grade level, and test description information 
should be transferred from the worksheet to Form R-3. Then fill in the 
approximate percent of the project that occurred between the pre- and 
posttest. The total number of children given Title I reading or math 
instruction in each grade level (this is not necessarily the same as 
the number tested) should be recorded in the appropriate spaces on each 
form. 

The number of children at each grade level having both pretest 
and posttest scores should be counted from the worksheets and recorded 
on the impact data form. Using the worksheet data, the ave;rage standard 
score on the pretest and posttest should be calculated for each grade 
level for reading and/or mathematics, but only for those children with 
both pre- and posttest scores. These means should be recorded on Form 
R-3 in the appropriate blanks. The percentile equivalents of these 
average standard scores should be found using the conversion tables in 
the test publisher's manual. Record the pretest and posttest percentile-. 
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scores on tha form. Do not convert each project child's standard score 
to a percentile score and then average the percentile scores to obtain 
the project mean. Finally, any differences between the number of children 
having both pre- and posttest scores and the number of children given 
instruction in the project should be listed for each grade level and 
program area i.e., X(number) dropped out of the project, Y(number) were 
absent for the pretest, etc. 

Finally, place all forms to be sent to the LEA in order and 
number them using the spaces provided in the top right corner. 
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Instructions for 
LEA Evaluation Report 



Orientation to the Reporting Forms 

Each form to be filled out by the LEA has a fprm number centered 
at the bottom of the page. The form number will begin with an R (for 
report) or a W (for worksheet). Forms with W-numbers are provided for 
the convenience of LEA staff and will not normally be sent to the SEA. 
Sheets which are not to be sent on are shaded along the right margin 
that they can be easily separated from report_Jpxms . Forms which are 
received by the LEA from projects, "schools, or districts will also be 
identified by R-numbers. These forms will remain at the LEA. (Form num- 
bers in a given model «^ill not necessarily be consecutive.) 

At the upper right of some forms, under the spaces for page num- 
bering, forms which are specific to one evaluation model are identified 
according to model (A, Bl, B2, CI, or C2). The forms are a'lso color 
coded according to model to make sorting by model easier if this is neces- 
sary. Forms which do not show a model identification at the upper right 
are general for all models. These forms should be completed for all 
projects in the LEA even if none of the evaluation models apply to the 
projects. (These general forms are R-4, R-6, and R-7- Form R-5 is also 
general to all models and should be completed by each Title I participating 
school or district.) 

If buildings in the LEA have used more than one reporting model, 
the building forms which are specific to one model should be sorted by 
model, and data from each building should be analyzed and reported by the 
LEA according to the instructions- for that model. 

General Information (Form R-4) 

The name of the LEA or the LEA number should be entered on the first 
line of Form R-4. The "number of buildings having Title I projects in the 
LEA should be listed on the second line. 
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Cost 

The Title I costs, other costs, and total costs for each building 
should be summed and recorded in the appropriate spaces under cost infor- 
mation. These data can be summed from the building Forms R-l. 

Participation 

Using the Form R-1 sent in by each building, locate the number of 
public and nonpublic school children served and the total number of chil- 
dren served in each grade level for each building, Thes^ data should be 
summed across all buildings in the LEA, and the result should be recorded 
to produce a table showing an unduplicated count of public, nonpublic, and 
total participation by grade level' for the entire LEA. 

The numbers of children participating in each program component in 
each building in the LEA should be added together and recorded in the 
second table. 

Evaluation 

The number of buildings reporting achievement test data should 
be reported and compared with the total number of Title I buildings exist- 
ing in the LEA. If these numbers are different, the reasons for the 
discrepancy should be listed, i.e., X(number) buildings conducted sub- 
jective evaluations, Y(number) buildings were concerned with affective 
areas only, etc. A breakdown of the number of buildings using each type 
of evaluation model should also be reported. Model A is the Norm- 
referenced Model which uses standardized achievement test data. Model 
Bl is the Control Group Model (with or without covariance) using stan- 
dardized achievement test data and Model B2 is the Control Group Model 
(with or without covariance) using criterion- or objectives-referenced 
tests. Model CI is the Special Regr.ession Model using standardized 
achievement tests and Model C2 is the Special Regression Model using 
criterion- or objectives-referenced tests. Thus, the design used in 
each evaluation report can be classified according to the following 
table. 
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Type of Data 


Type of Design 


Norm-referenced 


Control Group 


Special Regression 


Standardized . 
Achievement Tests 


Model A 


Model Bl 


Model Cl 


Objectives- or 
Criterion- 
Referenced Tests 




Model B2 


Model C2 



School or District Level Parent Advisory Councils (Form R-5) 

This form should be filled out for each Title .1 building and each 
Title I district in the LEA. Fill in the LEA name or number at the top 
of the form, then check the appropriate box (building or district) and enter 
the identifying information in the corresponding blank. Then fill in the 
requested information about the PAC in the spaces provided on the form. 

Parent Advisory Council Information (Form R-6) 

This form is a summary of all information submitted to the LEA on 
Forms R-5. Separate the Forms R-5 into two groups, one for buildings and 
one for districts. Fill in the LEA name or number at the top of the form, 
then enter the, total number of buildings in the LEA that participate in 
Title I. Then enter the number of building-level PACs in the LEA. (This infor- 
mation can be found on the Forms R-5.) Any discrepancies between these 
two numbers should be explained by building . 

^ The second section of the form is filled out in the same way, 

using the Forms R-5 which were filled out by districts. If each district 
does not have a PAC, an explanation should be given for each district 
if there is more than one district in the LEA. 

To fill out the last section of the form, combine the building 
and district Forms R-5. Examine each form to determine whether or not a 
majority of the PAC members in that building or district are parents of 
Title I children, and keep a tally to show for how many PACs these 
parents constitute a majority. Enter this total in the appropriate blank. 
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Then count the nximber of buildings or districts that checked each bdx on 
Form R-5 and enter each total number in the corresponding blank, on Form 
R-6. Then indicate the types of assistance the LEA gives local PACs by 
checking the appropriate boxes at the bottom of the form. 

Personnel and Training (Form R-7) 

Fill in the name or number of the LEA at the top of the form. To 
fill out the rest of the page, refer to all the Personnel and Training 
forms submitted by buildings (Forms R-2) • 



projects in each blank and ent?§r the sum in the corresponding blank on 
Form R- 7. It may be desirable to check project entries under "Other 
Personnel" to determine whether these personnel should be included in 
one of the other categories present on the form. 



made by projects in each box on Form R-2 and enter the total in the 
appropriate blank. Repeat this process for the last section of the form, 
which relates to days of training. 

Building Information (Form R-8) 

A Form R-8 should be filled out for every Title I building in the 
LEA. If a building's evaluation does not fit any of the reporting models, 
all information except achievement data^should be provided • Fill in the 
name or number of the LEA at the top of each sheet. From the Building 
Report Form R-1, copy the building name or nximber, the session during 
which the project operates, and the hours of project exposure per week. 
To calculate the total hours per project, multiply the hours of exposure 
per week by the total yeeks of project duration (also found on Form R-1). 
The instructor : pupil ratio should be copied from Form R-1. The unduplicated 
total number of children served is found on the total line of the last 
column under "Number of Children" on Form R-1. This should represent an 
unduplicated total of all the public and nonpublic school pupils at all 
grade levels served in the building. Copy the project cost information 
from Form R-1. To derive the project cost per pupil, divide the total 
project cost, by the number of children served. 



To complete the first part of the form, add the numbers entered by 




For the next section of the form, count the number of checks 
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The following procedures should be carried out for each building 
reporting achievement data for reading and/or mathematics. If a building 
reports both reading and mathematics data, fill out two forms. 

Step 1. Transfer the average pre-- and posttest percentile scores foir 
each grade from the Forms R-S submitted by the building to the appropriate 
columns on Form R-8» (Do not compute average percentile scores for grade 
bands 7-8 and 9-12.) 

Step 2. Using the Normal-curve Equivalent table included in Appendix C 
look up each percentile score and find its normal curve equivalent (NCE) . 

Step 3. Record these NCEs in the appropriate columns. 

Step 4. Compute a weighted average pretest and posttest NCE for grades 
7 and 8 and for grades 9-12. The weighted average is found by multiplying 
the number of children tested in a grade by the NCE for that grade, 
summing these products, and then dividing that sum by the total number 
tested in all grades involved. A workisheet for performing this computa- 
tion can be found in Appendix E. 

Step 5. Subtract the pretest NCE from the posttest NCE at each grade 
level to find the normal curve equivalent gain. Enter the gain in the 
last column on the form. 

Each building* s achievement data should be reported in this format 
using one or two forms for each building, depending on whether test scores 
are reported for one or two subject areas. 

Check the testing dates and make-up dates to be sure that make-up 
tests were administered within two weeks of the original tests. Also 
check to be sure that at least 80% of the project occurred between the 
pretest and the posttest. If either of these conditions are not met, 
include a note on the form to instruct the SEA that the data should not 
be included in a statewide aggregation. 

Buildings with sizeable discrepancies between the number of children 
tested and the number of children served should be examined carefully. The 
explanations for the discrepancies provided on Form R-3 should be analyzed. 
If the data sample appears not to be representative of the project or is 
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probably systematically biased, a note should be included on the form 
to instruct the SEA that that building data should not be included in 
the statewide aggregation. The factor to consider in determining whether 
the data sample is biased is not only the number or percentage of students 
missing from the data but also the extent to which the children in the 
data sample are representative of the group served. For instance, if 
some students failed to take the ppsttest because they graduated out of 
the project after reaching a certain level of achievement, the remaining 
sample is systematically biased. The same would be true if the students 
were removed from the project because they did not appear to be benefiting 
from it. If data are present for more than &5% of the project participants 
the data can be considered representati'^e. If" data are present for be- 
tween 70% and 85% of the students, check the explanation for the discrep- 
ancy to determine whether the sample is biased. If data are available 
for fewer than 70% of the students, the SEA should be instructed that the 
data are not suitable for aggregation. , 

When all the necessary forms have been completed, place all the 
forms in order and number them using the spaces provided in the top right 
hand corner of each page. • 
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GENERAL INFORMATION 



State 



page 1 of 



Number of . buildings with Title I projects operating in state 
COST 

Total Cost (for all projects in state) 



Title I funds Other 

PARTICIPATION (for all projects in state) 

Number of Children 

Grade Levels CUnduplibated Count) 

Served Public Nonpublic , Tctal 

Pre X 

K 

1 

2 

3 I ' 

4 ^^3^ 

5 — 

6 

7 — 

8 ^ ZIZZI 

9 

10 

11 " 

12 ; ^^^2 

Total 



Number of Children Served 
Program Components , (Duplicated Count) 

Reading/Lang. Arts/Communication 

M^th ' . . ~ _ 

Other Academic 

Affective ^ 
Support Services - -'"^ 



EVALUATION 

Number of Title I buildings reporting achievement test data 

If number of buildings reporting achievement test data is different from the 
total number of Title I buildings operating in the state, please explain. 



Number of buildings reporting achievement data by: 

Model A Model Bl Model B2 Model CI Model C2 
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Parent Advisory Council Information 



page of 



State 

Number of Title I funded .buildings 

Number of building-level PACs 

If the number of PACs is different from the number of buildings, please 
explain: 



Does each Title I participating district have a district-level PAC? 

Yes No 

If no, please explain: 



Total number of PACs (including districts-level PACs) that have a majority 
of members who are parents of Titl^e-'^T'children 




Total number ot-PACs that engaged in the following activities: 
Planned projects. 

Reviewed periodic progress reports. 

Reviewed evaluations of current and previous Title I projects. 

Informed and consulted parents about Title I services. 

Advised the LEA about the Title I application. 

Total number of LEAs which provided each of the following types of informa- 
tion and/or assistance to- their PACs: 

Copies of the Title I Act, Federal and State regulations and 

guidelines. 

Copies of the LEA's current application. 

Progress reports of Title I projects. 

Orientation and training materials. 

Plans for future Title I projects. 

Needs assessment data from Title I projects. 

Evaluation results of prior Title I projects. 

Procedures for responding to complaints and suggestions from 

parents. 

■ Funds provided to promote PAC activities. 
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PERSONNEL AND TRAINING 



page 



of 



Number and Type of Personnel Employed in Title I Projects Statewide 



Type of Personnel 
Instructional 

Teachers ^^^^.^-^ 

Instructional Aid^S'-'dr' Assistants 

Readin^-^Speci alis t s 
--'^fiat hematics Specialists 

Directors or Coordinators 

Resource Personnel 

Other Instructional Personnel 
Supportive 

Counselors or Psychologista 

Other Supportive Personnel 



Number XgidJ^ime Equivalent) 



Number of Buildings whose Personnel Received Training in Specified Topics 



Topic 

Orientation to Title I Project 

Project Planning and Design 

Subject Matter Areas (Reading, 
Math, Language Development, etc.) 

Use and Duties of Teacher Aides 

Use of Supportive Services 
(Counseling, Medical, etc.) 

Dissemination of Research Results 

Evaluation Design 

Other 



Teachers Aides 
(Number of Buildings) (Number of Buildings) 



Number of Buildings whose Personnel Received Training within Specified Ranges 
of Days 



Teacher Training 
(Number of Buildings) 



Aide Training 
(Number of Buildings) 
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page 
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A 



LEA Name or Number 



Building Name or Number 

□ Regular school year 

□ Summer 

□ Yearlong 

Hours of student project exposure per week 

Total hours per project 

Instructor: pupil ratio | ' 

Number of children served (unduplicated) 

Cost per pupil 

f. Reading 



Grade 


N 


Percentile 


Normal Curve 
Equivalent 


Normal Curve 
Equivalent 
Gain 


Pre 


Post 


Pre 


Post 


Pre K 














K 














1 














2 














3 














4 














5 














6 














7-8 














9-12 















Total N = 



Weighted Average Gain 



Grade 


N 


Percentile 


Normal Curve 
Equivalent 


Normal Curve 
Equivalent 
Gain 


Pre 


Post 


Pre 


Post 


Pre K 














K 














1 














2 














3 














4 














5 














6 














7-a - 














9-12 















Total N = 



Weighted Average Gain 
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LEA Name or Numbef 



Building Name or Number - 

D Regular school year 
D Summer 
□ Yearlong 

Hours of student project exposure per week 

Total hours per project 

Instructor : pupil ratio | 

Number of children served (unduplicated) 
Cost per pupil ' 



Reading 



Grade 


T 


Pretest 
Percentile 


Normal Curve 
Equivalent 
Gain 


Pre K 








K 








1 








2 








- 3 








4 • 








5 








6 








7-8 








9-11- 1 







Total N = 



Weighted Average Gain 
Mathematics 



ERIC 



Total = 



Grade 


T 


Pretest 
Percentile 


Normal Curve 
Equivalent 
Gain 


Pre K 








K 








1 








2 








3 ' 








4 , 








5 








6 








7-8 








9-12 









Weighted Average Gain 
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B2 



LEA Name or Number ^ 

Building Name or Number 

D Regular school year 
n Summer 
D Yearlong 

Hours of student project exposure per week 

Total hours per project 

Instructor : pupil ratio _: 

Number of children served (unduplicated) _ 
'Cost per pupil 



Reading 



Grade 


T 


T and C 
Standardized Test 
Percentile 


Normal Curve 
Equivalent- 
Gain 


Pre K 








K 








1 








2 








3 








4 








5 








6 








7-8 








9-12 









Total N^ = 

Weighted Average Gain 
Mathematics 



Grade 


T 


T and C 
Standardized Test 
Percentile 


Normal Curve 
Equivalent 
Gain 


Pre K 








K 








1 








2 








3 








4 








5 








6 








7-8 








9-12 









Total N^ = 

Weighted Average Gain 
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LEA Name or Number ^ 

Building Name or Number . 

□ Regular school year 

□ Summer 

□ Yearlong 

Hours of student project exposure per week 

Total hours per project 

Instructor :pupil ratio : 

Number of children served (unduplicated) _ 
Cost per pupil 



Reading 



Grade 


T 


Pretest 
Treatment %ile 


Normal Curve Equivalent Gains 


Means 


Intercepts 


Pre K 










K 










1 










2 










3 










4 










5 










6 










7-8 










9-12 











Total N^ = 

Weighted average gain (means) 

Weighted average gain (intercepts) 



Mathematics 



Grade 




• Pretest 
Treatment %ile 


Normal Curve Equivalent Gains 


M^ans 


Intercepts 


Pre K 










K 










1 










2 










3 










4 










5 










6 










7-8 










9-12 











Total N = 

^ Weighted average gain (means) 

Weighted average gain (intercepts) 
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C2 

LEA Name or Number 

Building Name or Number .. ^ ^ 

D Regular school year 

□ Summer 

□ Yearlong 

Hours of student project exposure per week - 

Total hours per project 

Instructor;pupil ratio _: 

Number of children served (unduplicated) 

Cost per pupil 



Reading 



Grade 


N- 
T 


Standardized Test 
Treatment %ile 


Normal Curve Equivalent 
Gains 


Means 


Intercepts 


Pre K 










K 










1 










2 










3 










4 










5 










6 










7-8 










9-12 











Tot4l N^ = 

Weighted average gain (means) 

Weighted average gain (intercepts) 



Mathematics 



Grade 


T 


Standardized Test 
Treatment %ile 


Normal Curve Equivalent 
Gains 


Means 


Intercepts 


Pre K 










K 










1 










2 










3 










4 










5 










6 










7-8 










9-12 











Total N^ -= 

Weighted average gain (means) 

Weighted average gain (intercepts) 
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Statewide Reading Achievement Data for Regular 
School Year and Yearlong Projects 



as a 


Function of 


Grade Level 


^ — 

Grade 


F-^ 

N 


Normal Curve 
Equivalent 
Gain 


Pre K 






K 






1 






2 






3 






4 






5 






6 






7-8 






9-12 







Statewide Mathematics Achievement Data for Regular 
School Year and Yearlong Projects 
as a Function of Grade Level 



Grade 


N 


Normal Curye 
Equivalent 
Gain 


Pre K 






K 






■ 1 






2 






3 






4 






5 






6 






7-8 






9-12 
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Number of Buildings Reporting Reading Gains Within Specified 
Ranges as a Function of Project Characteristics 



Normal Curve Equivalent Gain Ranges 





<0 


0-1,4 


1*5-2,8 


2,9-4.2 


4,3-5,6 


5,7-7,0 


> J ,\J 


Hours per week 
0-1.0 
















1,1-2.5 
















2, 6-5, 0 
































7,6-10,0 
















over 10,0 
































Total hours 
0-30 
















31- 60 
















61- 90 
















91-120 
































151-180 
















181-210 
















over 210 
















Cost per pupil 
0- 99 
















100-199 
















200-299 






• 










300-399 
















400-499 . 
















500-599 
















over 600 
















Ins true to r : pup il 
ratio 

1:>20 
















1:10.1 - 1:20 
















1:5.1 - 1:10 
















1:4.1 ■- 1:5 
















1:2.1 - 1:4 . 
















1:1.1 - 1:2 
















>l:l - 1:1 
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page of 

Number of Buildings Reporting Mathematics Gains within Specified 
Ranges as a Function of Project Characteristics 



Normal Curve Equivalent Gain Ranges 





<0 


0-1.4 


1.5-2.8- 


2.9-4.2 


4.3-5.6 


5.7-7.0 


>7.0 


Hours per week 
0-1.0 
















1, 1-2. 3 
















o ^ c rk 

2.6-3.0 
















3.1-7.3 
















7.6-10.0 
















over 10,0 
































Total hours 
0- 30 
















31- 60 
















61- 90 
















91-120 
















TOT T C O 

121-150 
















151-180 
















181-210 
















over 210 
































Cost per pupil 
0- 99 
















100-199 
















200-299 
















300-399 
















400-499 
















500-599 










/ 






over 600 
































Instructor : pupil 
rat io 

l:>20 
















1:10.1 - 1:20 
















1:5.1 - 1:10 
















1:4.1 - 1:5 
















1:2.1 - 1:4 
















1:1.1 - 1:2 
















>1:1 - 1:1 
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Instructions for 
SEA Evaluation Report 

General Information (Form R-19) 

Enter the name of the State on the first line of Form R-19. The 
number of Title I buildings operating in the State should be listed on the 
second- line. This number is found by summing the number of Title I 
buildings operating in the LEAs from Form R-4 of the LEA reports. 

Cost 

The total cost. Title I funds ^ and other funds used for all the 
projects should each be summed across LEAs using the figures provided on 
Forms R-4 in the LEA reports. These sums should be recorded under cost 
information on Form R-19. 

Participation 

The number of public and nonpublic school children served and. the 
total number of children served in each grade level should be located on 
Forms R-4 received from each LEA. Sum each of these data elements across 
all LEAs in the State. Record each sum on the SEA report Form R-19. The 
table provides an unduplicated count of public, nonpublic and total par- 
ticipation by grade level for the entire State. 

The number of children participating in each program component in 
each LEA should be added across the LEAs and recorded in the second table. 
These numbers represent a duplicated count of the total number of children 
in the State receiving each type of program. 

Evaluation 

The total number of buildings reporting achievement test data should 
be determined from the information in the LEA reports on Forms R-4 and re- 
ported on Form R-19. This number should be compared With the total number 
of Title I buildings in the state (on line 2, Form R-19). If these 
figures are different, the reasons for the discrepancy should be listed, ^ 
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i.e.', X (nximber) buildings were late reporting data^vy (number) buildings 
focused on affective areas, etc. Primarily, this should be a summary of 
the reasons given by the LEAs for not reporting data for all of their 
buildings. 

A breakdown of .the number of buildings in the State using each type 
of evaluation model should be reported at the bottom of Form R-19. This 
information' can be* computed by summing the numbers provided in, the LEA 
evaluation reports on Forms R-8 under each model type. Model A is the 
Norm-referenced Model which uses standardized achievement test data. Model 
Bl is the Control Group Model (with or without covariance) using standard- 
ized achievement test data. Model B2 is the Control Group Model (with or 
without covariance) using objectives- or criterion-referenced tests. Model 
CI is the. Special Regression Model using standardized achievement tests and 
Model C2 is the Special Regression Model using objectives- or criterion- 
referenced tests. The designs used in each evaluation report were classi- 
fied by the LEA according to the following table. 



Type of Data 


Type of Design 


Norm-referenced 


Control Group 


Regression 


Standardized 
Achievement tests 


Model A 


Model Bl 


Model CI 


Objectives- or 
Criterion- 
Referenced Tests 




Model B2 


Model C2 



. Parent Advisory Council Infonnatibn (Form R-20) 

Using the information in the LEA reports on Forms R-6, sum the num- 
ber of Title I buildings operating in dll the LEAs and sum the number of 
building-level PACs. Record these numbers on Form R-20. Any discrepancy 
between these numbers should be explained. 



If every LEA indicated on Form R-6 that it had a district level 
PAC, then check "Yes" on Form R-20. If one or more LEAs indicated that 
they did not have a district level PAC, check "No" on Form R-20 and explain 
the reasons by district. 




Compute the total number of PACs that have a inajority of members 
who are parents of Title I children by summing the xiumbers provided on 
Forms R-6 in the LEA reports. Record this number in the appropriate place 
on Form R-20. 

Sum the number of PACs that engaged in each of the listed activi- 
ties using the figures reported on Forms R-5» Record these numbers on 
Form R-20. 

To fill out the last section of Form R-20, keep a tally of the 
number of LfiAs that indicated that they provided the types of information 
or assistance listed. Enter the totals in the appropriate blanks on 
Form R-20. 

Personnel and Training (Form R-21) \ 

To fill out Form R-21, refer to all the Personnel and Training 
forms submitted by LEAs (Forms R-7). Sum the numbers entered on Forms 
R-7 for numbers of each type of personnel, topics covered during training, 
and days of training received. Enter these sums in the appropriate 
places on Form R-21. 

Building Information (Forms R-22, R-23. R-24, R-25, and R-26) 

A building information sheet is filled out for each Title I 
building in the state. The information is recorded on one of five possible 
forms. The proper form is determined by the type df evaluation model used 
by the building. If a building submitted no achievement data, the summary 
information should still be reported. If a building conducted a Model A 
evaluation, fill out Form R-22; if Model Bl, use Form R-23; if Model B2, 
use Form R-24; if Model CI, use Form R-25; and if Model C2, use Form R-26. 
To simplify the transfer of information, the building information forms 
have been color coded. Each of the five evaluation models uses a form of 
a different color. The same color form that was used to report on the 
building by the LEA should be used by the SEA. If a building reported no 
achievement 'data, transfer the summary building characteristics to an SEA 
level form of the same color. 
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Most of the. necessary achievement data can be obtained from the 
LEA reports and only summary calculations will have to be computed at the 
state level. In general, the summary calculations will include: the 
averaging of data across grade bands 7-8 and 9^12, the calculation of the 
total number of children tested in Reading and/or Mathematics in a buULding, 
the weighted average gain for the building in Reading and/or Mathematics, 
and weighted average gains at each grade level across buildings in the 
state. 
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Model A Evaluation (Form R-22) 

Begin with a building's Reading achievement data. 

Step 1, Transfer the summary building characteristics from Form R-8 in 
the LEA report to Form R-22. This ''information should include the LEA and 
building identifiers, whether it is a yearlong, summer, or regular school 
year project, hours of project exposure per week, total hours per project, 
instructor: pupil ratio, number of children served and cost per pupil. 

Step 2. Transfer the impact data to Form R-22 for all grades and grade 
bands. This includes the Ns (from Column 1, Form R-8), pre and post 
percentiles (from Columns 2 and 3, Form R-8), pre and post normal curve 
equivalents (from Columns 4 and 5, Form R-8), and normal curve equivalent 
gains (last column. Form R-8) for each grade level. 

Step 3. Determine the total N by summing the numbers in the N column on 
Form R-22 and record this number in tHe appropriate blank. 

Step 4. Calculate a weighted average gain for the project in Reading/ 
Mathematics, using the worksheet provided in Appendix E, and enter this 
figure in the appropriate blank on Form R-22. 

Step 5. Repeat Steps 2 through 4 for the building's Mathematics data. 

If the building has no Mathematics data, go to Step 6. 

Step 6. Repeat Steps 2 through 5 for all buildings using a Model A 
evaluation. 
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Model Bl Evaluation (Form R-23) 

Begin with a building's Reading data. 

Step 1. Transfer the summary project characteristics from the top of 
Form R-10 in the LEA report to Form R-23. 

Step 2. For Pre K through grade 6, transfer the following achievement 
information from Forms R-10 to R-23: N^s (from Column 1, Form R-10), the 
treatment groups pretest percentile score (from Column 9, Form R-10), and 
the normal curve equivalent gains (from the last column. Form R-10). 

If no children were served in grades 7-12, skip to Step 6. 

Step 3. For grade bands 7-8 and 9-12, calculate N^s by summing the number 
of children tested at each grade level for each grade within the band. Use 
the numbers found in Column 1 of Form R-10. Record the N^s on Form R-17. 

Step 4. For grade bands 7-8 and 9-12, calculate a pretest percentile for 
each band. 

a. Using the worksheet in Appendix E, compute a weighted average 

. pretest NCE for each grade band (using pretest NCEs from Column 
15 , Form R-10) . 

. b. Convert the weighted average pretest NCE to a percentile score 

using Appendix D and record the percentile on Form R-23. 

Step 5. For grade bands 7-8 and 9-12, calculate a weighted average normal 
curve equivalent gain for each band, using the instructions and worksheet 
in Appendix E. The Ns and average NCEs are found in the first and last 
columns of Form R-10. Enter the weighted averages on Form R-23. 

Step 6. Compute the Total N^ by summing the numbers in the N^ column on 
Form R-23. Record it in the appropriate space. 

Step 7. Compute a weighted average normal curve equivalent gain for the 
project in Reading/Mathematics using the worksheet provided in Appendix E, 
and enter this figure in the appropriate space. 

Step 8. Repeat Stepts 2 through 7 for the Mathematics data of the building. - 

If the building has no Mathematics data, go to Step 9. * 
Step 4. Repeat Steps 2 through 8 for all buildings using a Model Bl evaluation. 
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Model B2 Evaluation CFdrm R-24) 

Begin with a building's Reading data. 

Step 1. Transfer the summary project characteristics from the top of 
Form R-12 of the LEA reports to Form R-24. 

Step 2. Transfer the impact data from Form R-12 to Form R-24 for Pre K 
through grade 6,/ This includes the N^s (from Column 1, Form R-12), needs 
assessment percentiles (from Column 4, Form R-12) and the NCE gains (from 
the last column, Form R-13). 

If no children were serv^d^^n grades 7-12, skip to Step 6. 

Step 3. For grade^^aiid:g:^8 and 9-12, calculate by summing the number 
of children^jte^rted at each grade within the band (from Column 1, Form R-12) 
and record them on Form R-24. 



Step 4. For grade bands 7-8 and 9-12, calculate a needs assessment percen- 
tile for each band. 

a. Locate the treatment and control groups' needs assessment per- 
centile for each grade level in the band (from Column 4, Form 
R-12). 

b. Convert each percentile score to its normal curve equivalent 
using Appendix C. 

c. Using the worksheet in Appendix E, compute a weighted average 
NCE for the grade band, using as N the sum of N and N at 
each grade level (from Columns 1 and 2, Form R-12). 

d. Convert each NCE score to its percentile equivalent usjLn^ 
Appendix D and record the numbers on Form Jl-24. 

Step 5. For grade bands 7-8 and 9-12, c^eulate^a weighted average normal 
curve equivalent gain for each.gra-ae band. Use the worksheet in Appendix 
E, and record the number ^in the appropriate space on Form R-24. (N's can 
^ be found in Column 1 on Form R-12 and NCE gains in Column 5 of Form R-13.) 

Step 6. Determine the total N^ by summing the numbers in the N^ column 
on Form R-24 and record it. 
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Step 7. Compute a weighted average NCE gain for the building in Reading/ 
'MathepatiTcs, using the worksheet in Appendix E. Enter thiis figure in the 
appropriate space. 

Step 8. Repeat Steps 2 through 7 for the Mathematics data- of the building. 
If the building has no Mathematics data, go to Step 9, 

Step 9. Repeat Steps 2 through 8 for all buildings using a Model B2- . 
evaluation,^ ^^^-^^^^'^ ' 
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Model Cl Evaluation (Form R-25) 

Begin with a building* s Reading data. 

Step 1. Transfer the summary projebt characteristics from the top of 
Form R-15 of the LEA report to Form R-25. 

Step 2. Transfer the impact data from Form R-15 to Form R~25 for Pre K 
through grade 6. This includes the N^s (from Column 1, Form R-15), the 
pretest treatment percentiles (from Column .3, Form R-15) and the means 
and intercepts (from the last two columns, Form R-15). 

If no children were served in grades 7-12, skip to Step 6. 

Step 3. For grade bands 7-8 and 9-12, calculate N^s by summing the number 
of children tested at each grade within the band (from Column 1, Form R-15) 
and record the numbers on Form R-25. 

Step 4. For grade bands 7-8 and 9-12, calculate a pretest treatment per- 
centile for each band. 

a. Locate the pretest treatment percentile for each grade level 
in the band (from Column 3^, Form R-15). 

b. Convert each percentile score to its normal curve equivalent 
using Appendix C. 

c. Using the worksheet in Appendix E, compute a weighted average 
pretest NCE for each grade band. 

d. Convert each of these NCE scores to its percentile equivalent 
using Appendix D and record the numbers on Form R-25. 

Step 5. For grade bands 7-8 and 9-12, calculate weighted average normal 
curve equivalent gains for the means and again for the intercepts for each 
band, using the worksheet in Appendix E. The N*s are found in Column 1 of 
Form R-15, the Gains (means) in Column 8, and the Gains (intercepts) from 
the last column. Record the results in the appropriate spaces on Form 
R-25. 

Step 6. Determine the total by summing the numbers in the column on 
Form R-25 and record this number on Form R-25. 
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Step 7. Compute two weighted average NCE gains for the building in 
Reading/Matheinatics, using the worksheet in Appendix E. One NCE gain 
will be. based on the means and the other will be based on the intercepts. 
Record the figures in the appropriate spaces on Fofm 

Step 8. Repeat Steps 2 through 7 for the Mathematics data of the building. 
If the building has no Mathematics data, go_^to,..Step. 9, 

Step 9. Repeat Steps 2 through 8 for all buildings using a Model CI 
evaluation. 
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Model C2 Evaluation CFonn R-26) . 

Begin with a building^s Reading data. . • 

Step 1. Transfer the summary building characteristics from the top of 
P^rm R-18 of the LEA reports to Form R-26. 

Step 2. Transfer the impact data from Form R-18 to Form R-26 for Pre K 
through grade 6. This includes the N^s (from Column 1, Form R-18) , the 
standardized test treatment percentiles (from Column 2, Form R-18) and 
the NCE mean and intercept gains (from the last two columns on Form R-18). 

If no children were served in grades 7-12, skip to Step 6. 

Step 3. For grade bands 7-8 and 9-12, calculate by summing the number 
of children tested at each grade within the band (from Column 1, Form R-18) 
and record the numbers on Form R-26. 

Step 4. For grade bands 7-8 and 9-12, calculate a standardized teat 
treatment percentile for each band. 

a. Locate the standardized test treatment percentile for each 
grade level in the band (from Column 2, Form R-18). 

b. Convert each percentile score to its normal curve equivalent 
using Appendix C. 

* c. Using Appendi-x E, compute a weighted average NCE for the 
grade band. 

d. Convert each NCE score to its percentile equivalent using 
Appendix D and record the numbers on Form R-26. 

Step 5. For grade bands 7-8 and 9-12, calculate weighted average normal 
curve equivalent gains for the means and again for the intercepts for each 
band, using the worksheet in Appendix E. The N*s are found in Column 1 
of Form R-18, the Gains (means) in Column 3, and the Gains (intercepts) in 
Column 4. Record the results in the appropriate spaces on Form R-26. 

Step 6. Determine the total N^ by summing the numbers in the N^ column 
on Form R-26 and record the sum in the appropriate place. 

Step 7. Compute two weighted avetage NCE gains for the project in Reading/ 
Mathematics, using the worksheet in Appendix E. One NCE gain will be based 
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on the means and the other will be based on the intercepts. Record the 
numbers in the appropriate spaces on Form Rt-26. 

Step 8, Repeat Steps 2 through 7 for the Mathematics data of the building. 
If the building has no Mathematics data, go to Step 9. 

Step 9, Repeat Steps 2 through 8 for all buildings using a Model C2 
evaluation. 
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Stat;ewide Reading and Mathematics Achievement Data (Form R-27) 

The tables on Form R~27 are to 3ummarize the achievement data for 
regular school year and yearlong projectls only. Data from summer projects 
are not to be included. The data that the LEAs have indicated as not 
suitable for ^aggregation should not be included. 

Begin with all the building reports that included Reading data. 

Step 1. Sum the number of children testec^ at each grade level or grade 
band across all buildings. Enter these niinbers in the appropriate spaces 
on the upper table. These figures will represent the number of children 
tested at each grade level/band in the statue. 

Step 2. .Calculate a weighted average NCE gain for each grade level or 
grade band, using the instructions and worksheet in Appendix E. Record 
the numbers in the upper table on Form R~27. 

< 

Step 3. Repeat Steps 1 and 2 for the buildings reporting Mathematics 
data^ Record tliis information in the lower table on Form R-27. 
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Relation between project gains and other project characteristics (Forms 
R-28 and R-29) 

Each of the tables relating project impact to other project 
characteristics. Forms R-28 and R'-29, should be filled out using the pro-, 
cedure described below. The easiest method is to fill out all the informa- 
tion on each building at one time rather than completing one form or table 
at a time. A running tally should be kept on a worksheet and the final 
results recorded on the tables provided. 

Begin with the buildings that used Model A for evaluation (Form 

R-22) . 

Step 1. Select one building's information sheet. 

Step 2. Locate its average weighted NCE gain in Reading. 

Step 3. On the table on Form R-28, find the range of NCE gains in which 
that project's gain fits. This determines the column in which all successive 
tallies will be marked. 

Step 4. Locate the hours of exposure per week for the project and determine 
the range in which it belongs on Form R-28. 

Step 5. Find the cell in which the row and column intersect and place a 
mark in that cell. 

Steps 6 through 11. Repeat Steps 4 and 5 for the total hours, cost per 
pupil, and instructor :pupil ratio of that project. 

Step 12. If the project had a Mathematics component , find the project's 
average weighted NCE gain for math. 

Steps 13 through 20. Mark the appropriate cells for hours per week, total 
hours, cost per pupil, and instructor :pupil ratio on Form R-29. 

Step 21. Repeat Steps 1 through 20 for all buildings using Forms R-22, R-23, 
R-24, R-25, and R-26. 

Step 22. Count the tallies per cell and record these numbers in the appro- 
priate places on the final version of the tables on Forms R-28 and R-29. 
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